qmail: move Plesk mailboxes to a new server

You can use the following when you have a Plesk installation on two servers and need to move mailboxes from one server to another one.

Note that it assumes that this is all done at a time when there is no activity on the system. If there is no such timeframe in your case, you’ll probably need to do a little more work making sure you do not lose any incoming email in the process.

First, I assume that the domain has already been created on the new server but the DNS entries still point to the old server. Now you need to create a script to create the mailboxes. This can be done by a simple SQL script run against the old psa database:

use psa;
SELECT CONCAT("/opt/psa/bin/mail --create ",mail . mail_name,"@",domains . name ," -cp-access true -passwd ",accounts . password," -mbox_quota -1 -mailbox true")
FROM `mail` , accounts , domains
WHERE
mail . account_id =accounts . id AND
mail . dom_id =domains . id AND
domains . name = "my-domain-name.com"
ORDER BY `mail` . `postbox`  ASC;

If Plesk is not installed in /opt/psa, you’ll have to replace it in the SELECT statement by the path to your Plesk installation e.g. /usr/local/psa.

This has to be executed in a mysql client e.g.:

mysql -uadmin -p`cat /etc/psa/.psa.shadow`

It will return something like:

/opt/psa/bin/mail --create doc@my-domain-name.com -cp-access true -passwd mypassword1 -mbox_quota -1 -mailbox true
/opt/psa/bin/mail --create post@my-domain-name.com -cp-access true -passwd mypassword2 -mbox_quota -1 -mailbox true

Here, we have two user mailboxes for this domain: doc and post.

Here some information regarding the options used:

  • –create: it creates a Plesk e-mail account
  • doc@my-domain-name.com: this is the email address you’ll create
  • -cp-access true: enables the mail name user to Plesk control panel
  • -passwd mypassword1: sets the same password as on the old server
  • -mbox_quota -1: Sets the size of the mail box for the mail name to the default mail box size value set for the domain to which the mail name belongs
  • -mailbox true: creates mail box for the mail name

Now the mailboxes also exist on the new server and you just need to copy the files from one server to another. I copy them through my local machine using cp but you could also use rsync instead or directly cp from one server to the other one. I execute the following on my local computer:

mkdir /tmp/qmail
cd /tmp/qmail

rm -rf cur new
scp -r root@myoldserver:/var/qmail/mailnames/my-domain-name.com/doc/Maildir/cur .
scp -r cur root@mynewserver:/var/qmail/mailnames/my-domain-name.com/doc/Maildir/
scp -r root@myoldserver:/var/qmail/mailnames/my-domain-name.com/doc/Maildir/new .
scp -r new root@mynewserver:/var/qmail/mailnames/my-domain-name.com/doc/Maildir/

rm -rf cur new
scp -r root@myoldserver:/var/qmail/mailnames/my-domain-name.com/post/Maildir/cur .
scp -r cur root@mynewserver:/var/qmail/mailnames/my-domain-name.com/post/Maildir/
scp -r root@myoldserver:/var/qmail/mailnames/my-domain-name.com/post/Maildir/new .
scp -r new root@mynewserver:/var/qmail/mailnames/my-domain-name.com/post/Maildir/

Replace myoldserver by the name or IP address of the old server and mynewserver by the name or IP address of the new server. I have configure both servers to accept login with a public key. If it is not the case for you, you’ll have to enter a few passwords.

If you have more than two mailboxes, you will need to repeat the five lines more than twice.

The new directory contains new emails not yet read and the cur directory contains emails already read.

After that, all you need is to make that the writes are fine. To do this execute the following on the new server:

chown -R popuser:popuser /var/qmail/mailnames/my-domain-name.com/doc/Maildir/cur
chown -R popuser:popuser /var/qmail/mailnames/my-domain-name.com/doc/Maildir/new

chown -R popuser:popuser /var/qmail/mailnames/my-domain-name.com/post/Maildir/cur
chown -R popuser:popuser /var/qmail/mailnames/my-domain-name.com/post/Maildir/new

After that change the DNS setting for mail.my-domain-name.com to point to the new server. The propagation of the DNS server is usually fast but can sometimes take many hours. It’s all a matter of luck…

Qmail: Error messages returned by the server when receiving emails

After a reinstallation of Plesk, the default mail server was changed from qmail to postfix. After switching back to qmail, it looked like qmail didn’t have a problem with the previously existing mailboxes but when sending emails to an email address hosted on this server a few problems occured.

The first time I sent a test Email, I got the following answer from the server:

553 sorry, that domain isn’t in my list of allowed rcpthosts; no valid cert for gatewaying (#5.7.1)

This basically means I need to add the domain name for this email address to /var/qmail/control/rcpthosts. rcpthosts is configuration file containing a list of domain names (one per line), for which qmail should accept emails. If you send an email to info@mydomain.com and mydomain.com is not in the list, qmail will refuse to accept the email. By default, qmail works as an open relay, i.e. you can contact this mail server and state an email address not hosted on it but on a third server and qmail will accept the email and relay it to the third server. So by default qmail will accept all incoming messages. This is of course not a very good idea. Also if a lot of Spam is relayed like this, your mail server will be penalized. So you need to have an rcpthosts file to make sure that qmail will only accept emails for specific domains hosted on this server.

Another interesting file (not really related to the error message but interesting) is /var/qmail/control/locals. This file is used to configure domains for which emails will be delivered locally. So if you add mydomain.com to this file and then send an email from the server itself to info@mydomain.com then this will directly go to the local queue.

Another relevant file is /var/qmail/control/virtualdomains. It allows to define virtual domains. You can define a user that will receive mail for a given domain. You can also define a 1:1 domain alias in there.

You might also get the following error message:

Sorry. Although I’m listed as a best-preference MX or A for that host, it isn’t in my control/locals file, so I don’t treat it as local. (#5.4.6)

As the message says, you will need to add the mail domain to the /var/qmail/control/locals additionally to the localhost. Also make sure that the fully qualified domain name is entered there and not only the hostname.

If any change doesn’t produce any result, it’s probably that you haven’t restarted qmail or have it reload the configuration.

Reloading the configuration is done with one of the following commands:

/etc/init.d/qmail reload
qmailctl reload
/var/qmail/bin/qmailctl reload
svc -h /service/qmail-send

Basically what it does is send a HUP signal to qmail-send.

To complete restart qmail, use one of the following:

qmailctl restart
/var/qmail/bin/qmailctl restart
/etc/init.d/qmail restart

You can also restart qmail by running these three commands in this order:

svc -d /service/qmail-smtpd
svc -t /service/qmail-send
svc -u /service/qmail-smtpd

If none of those work, try:

/etc/rc.d/qmail stop
/etc/rc.d/qmail start

Sometimes qmail-send needs more time to stop or just doesn’t stop. You can check it with:

ps -ef | grep qmail-send | grep -v grep

If it returns anything, it means qmail-send failed to stop. In this case, you’ll need to kill it before starting it again:

killall -9 qmail-send

You might also need to kill qmail-remote and qmail-local:

killall -9 qmail-remote
killall -9 qmail-local

If you are using Plesk you might find the qmail start/stop script there:

/usr/local/psa/rc.d/qmail restart
/opt/psa/rc.d/qmail restart

Qmail: 30 to 60 seconds connection delay

I’ve noticed that when connection to port 25 of one of our servers, it took quite a long time until commands can be sent. To test it, zou connect using telnet>

telnet xxx.xxx.xxx.xxx 25

Where xxx.xxx.xxx.xxx is the IP address of the server on which the mail server is running.

I saw that the connection was immediately established but it took quite some time for the server to send a 220 response. Before this response is received no commands (e.g. HELO, QUIT…) can be processed. Here’s the sequence in which things happen:

  1. Client attempts to open an SMTP connection to port 25 of the server.
  2. Client waits for the socket connection to be established.
  3. Connection established.
  4. Client waits for protocol to start (i.e. waits for a 220 message).
  5. Server sends a 220 code saying it is ready for action.
  6. Now the client can send commands to the server.

The delay occurs in step 4. It takes a long time until the server sends a 220 code.

What happens in the background is that the server validates that the client is trustworthy. This involves performing a reverse lookup of the IP address and checking for known spammers. This can take some time especially if the reverse lookup results in a timeout.

Here are a few things you need to do to get rid of this timeout:

First, make sure that the name servers that are listed in /etc/resolv.conf are working properly.

You can check whether the IP from where the client is accessing the server can be reverse resolved. You can check it with:

# host 192.168.1.1
Host 1.1.168.192.in-addr.arpa. not found: 3(NXDOMAIN)

If you get a “not found” as shown above then it means the name resolution will not work. You might also get the following:

# host 192.168.1.1
;; connection timed out; no servers could be reached

In this case, the reverse lookup times out. You can prevent the qmail from performing a reverse lookup by changing the xinetd script used to start qmail. The script is located in /etc/xinetd.d. It’s usually called smtp or smtp_psa if you’re using Plesk. It looks like this:

service smtp
{
       socket_type     = stream
       protocol        = tcp
       wait            = no
       disable         = no
       user            = root
       instances       = UNLIMITED
       server          = /var/qmail/bin/tcp-env
       server_args     = /usr/sbin/rblsmtpd  -r sbl-xbl.spamhaus.org /var/qmail/bin/relaylock /var/qmail/bin/qmail-smtpd /var/qmail/bin/smtp_auth /var/qmail/bin/true /var/qmail/bin/cmd5checkpw /var/qmail/bin/true
}

tcp-env supports the following options:

-r (Default.) Attempt to obtain TCPREMOTEINFO from the
remote host.

-R Do not attempt to obtain TCPREMOTEINFO from the remote
host.

-t
Give up on the TCPREMOTEINFO connection attempt after
timeout seconds. Default: 30.

So we need to use the -R option and could also set the timeout to 0 seconds with the -t0 option:

service smtp
{
       socket_type     = stream
       protocol        = tcp
       wait            = no
       disable         = no
       user            = root
       instances       = UNLIMITED
       server          = /var/qmail/bin/tcp-env
       server_args     = -Rt0 /usr/sbin/rblsmtpd  -r sbl-xbl.spamhaus.org /var/qmail/bin/relaylock /var/qmail/bin/qmail-smtpd /var/qmail/bin/smtp_auth /var/qmail/bin/true /var/qmail/bin/cmd5checkpw /var/qmail/bin/true
}

Actually -t0 is redundant when using -R. So you can try with -R and with -Rt0 and see whether it makes any difference on your system.

After changing this, you’ll need to restart xinetd:

/etc/init.d/xinetd restart

If you use inet instead of xinetd then you need to update /etc/inetd.conf instead and restart inetd by using:

/etc/init.d/inetd restart

or:

killall -HUP inetd

Please note that Plesk seems to overwrite this file once in a while. So you might loose this setting.

Even after setting this, I’ve seen that from most servers the delay was gone but I still had a remaining 30 seconds delay when connection to the mail server from my laptop.

The mail server uses to auth protocol to contact the client machine. Unfortunately, many firewalls are configured to just ignore these requests without answering. In this case, you’ll also have to wait for a timeout to occur before the mail server goes to the next step.

In order to get rid of it, you can configure a rule using iptables to actively reject connections from the server to a remote port 113 so that you do not have to wait for 30 seconds in case the remote firewall just silently ignores your request. The processing of the request on port 113 is not critical for the functioning of the mail server, it’s safe to prevent it. Execute the following on the server to have these connection rejected by the local firewall on the server:

iptables -A OUTPUT -p TCP --dport 113 -j REJECT --reject-with tcp-reset

Making these few changes made sure that the connection from our other servers was done without delays. And from my laptop I do not get a fix 60 seconds or 30 seconds delay anymore but it’s mostly taking between 1 and 15 seconds.

If after doing all this, it’s still slow, you should check the list of servers used by rblsmtpd. rblsmtpd blocks mail from RBL-listed sites. It uses a list of sources. In my case only sbl-xbl.spamhaus.org. First make sure that all source are still up and running, since this could also cause an additional timeout (the default timeout is 60 seconds). Additionally, checking whether the client is blacklisted also costs time. So you might want to remove rblstmpd and its parameters (basically what’s between -Rt0 and /var/qmail/bin/relaylock above and see whether it’s faster.

Empty qmail remote queue

A few weeks ago, we had some emails which got stuck in the remote queue of qmail. Our server kept trying to process them again but it never succeeded. After a few days I wanted to just empty the queue to stop seeing in the log files messages about failed delivery.

Unfortunately there seems to be no built-in function in qmail to empty a queue (although I’d love to be proven wrong). So I started looking for a solution how the get rid of these messages.

Since qmhandle was already installed on our server I tried using it to empty the queue. This didn’t work on our server but here’s how to do it (maybe it will work for you…):

Delete all messages containing a specific string in the subject:

#qmHandle -S"subject_substring_to_delete"

Delete all messages in both the local and remote queues:

#qmHandle -D

Since this didn’t work, I started looking for other solutions.

There’s a tool called Qmail-Remove. Unfortunately I wasn’t able to download it and it looks like it’s not really maintained anyway…

After some more searching, I finally found out that it’s pretty easy to empty the queue without any tool.

You just have to stop qmail with:

/etc/init.d/qmail stop

(if it doesn’t work try one of these:

/etc/rc.d/qmail stop
qmailctl stop

)

Then you just need to delete all files in the subdirectories of /var/qmail/queue/remote (or another path if your queues are stored somewhere else):

find /var/qmail/queue/remote -type f -exec rm {} \;

Instead of “remote” you can enter another subdirectory of “queue” to empty another queue e.g. local.

An then start qmail again:

/etc/init.d/qmail start

That’s it. Now you can check the queue status, with one of the following:

# qmHandle -s
# qmail-qstat
# qmail.qread
# qmailctl queue

They should all show that your queue is now empty !