What's the first thing you do when setting up as an indie gamedev? Why, set up a mailserver, of course! This is Part 2 of the process I went through, where we check incoming mail for spam and viruses...

If you're here from Part 1, welcome back. If not, you've joined in the middle of me documenting the rather lengthy journey I took from an empty server to one handling email with TLS, spam and virus checking, mail accounts handled in a database, and with SPF, DKIM, and DMARC checking and compliance.

Part 2 will cover using Amavis, SpamAssassin, and ClamAV to check incoming mail.

INCOMIIIIIIING! - Spam and Viruses

Once you've told the world that you have a mailserver and you're willing to let people email you, you're going to get a lot of people who want to email you. Some of them will be family, friends, or business contacts. But a lot of them will also be spammers who want you to buy a variety of questionable products, and compromised computers that would love to spread viruses and other malware far and wide.

In order to defend ourselves from this constant assault we're going to use mail filters (often called milters, yes, really) to scan incoming mail for viruses and to assess whether the mail looks like spam. For this we'll use Amavis, ClamAV, and SpamAssassin.

Amavis is an interface between the MTA (i.e. Postfix) and content checkers (in our case, ClamAV and SpamAssassin). Postfix hands the email over to Amavis, which gets various filters to check the mail. Assuming all is well, the mail gets passed back to Postfix, but it can be tagged with headers or even discarded if it looks dodgy or dangerous.

We may as well get all the packages we want for this now. On Ubuntu 18.04 LTS, for amavisd-new it's important that you get a version no lower than 1:2.11.0-1ubuntu1.1 which, at time of writing, means installing from the bionic-updates repository rather than bionic. bionic's version, 1:2.11.0-1ubuntu1, has a bug with the "originating" flag, which I'll explain in a later post when it becomes relevant:

apt-get -t bionic-updates install amavisd-new clamav-daemon spamassassin

This will probably want to install a lot of package dependencies, and will take up quite a lot of disk space.

Let's start with virus scanning. On Debian-based systems, Amavis config is spread across several files in /etc/amavis/conf.d. In other distros, you might get a big config file in /etc/amavisd/amavisd.conf, but I'm going to assume you're using the multiple file version - if you check /etc/amavis/conf.d/15-av_scanners you should see a lot of config options for various scanners. ClamAV is at the top, at least in the Ubuntu package, and should be uncommented by default. Amavis config files aren't pretty, but it should look like this:

### http://www.clamav.net/
 ['ClamAV-clamd',
   \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd.ctl"],
   qr/\bOK$/m, qr/\bFOUND$/m,
   qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],
# NOTE: run clamd under the same user as amavisd, or run it under its own
#   uid such as clamav, add user clamav to the amavis group, and then add
#   AllowSupplementaryGroups to clamd.conf;
# NOTE: match socket name (LocalSocket) in clamav.conf to the socket name in
#   this entry; when running chrooted one may prefer socket "$MYHOME/clamd".

Note the comments at the bottom. To add clamav to the amavis group, run

adduser clamav amavis

The ClamAV config file you need to edit is /etc/clamav/clamd.conf. For the LocalSocket setting, check that it matches the value in your Amavis config (here, /var/run/clamav/clamd.ctl).
The AllowSupplementaryGroups setting you can ignore, it's been deprecated.

Pleasingly, you don't have to do anything for SpamAssassin. Amavis is built using SpamAssassin's libraries, and spam detection is working out of the box. After you've been up and running for a while, you might want to tweak the thresholds that SpamAssassin uses for flagging things as spam to make it more or less aggressive, but we'll just run with the defaults for now.

Once that's done, head back over to Amavis's config and look in /etc/amavis/conf.d/15-content_filter_mode - there are two settings here for enabling virus scanning and spam detection. Slightly confusingly, we need to uncomment two bits of config called @bypass_virus_checks_maps and @bypass_spam_checks_maps. My version of this file was heavily commented to make it clear that yes, you uncomment the "bypass" stuff to enable checking. If you don't define the circumstances under which checks should be bypassed, they won't run at all. So, you should end up with the following uncommented lines:

@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

By doing this, you're telling Amavis "please check for viruses and spam, but check the following variables (e.g. the @bypass_spam_checks_acl array) to see when not to bother". Those variables are actually empty in a default installation, so we'll check everything.

Amavis should be set up to grab our hostname, which it needs much like Postfix did. Open up /etc/amavis/conf.d/05-node_id, and you should see a line that will call the hostname command to set its own idea of the hostname. If you needed to set it manually for some reason, there's a commented-out manual line below. The bit that should set it automatically is:

chomp($myhostname = `hostname --fqdn`);

So if that's there you don't need to do anything. We also need to tell Amavis what domains should be considered "local", so that it handles mail for example.com as though it's the final destination for that mail, which will include things like DKIM verification later. We do this by setting the @local_domains_acl. If you head into 05-domain_id, you'll see a similar thing to 05-node_id:

chomp($mydomain = `head -n 1 /etc/mailname`);
@local_domains_acl = ( ".$mydomain" );

This one isn't quite right. /etc/mailname, by default, will be mail.example.com, which means that Amavis will consider mail local only if it's addressed to, say, dave@mail.example.com (or a subdomain, so dave@something.mail.example.com). It won't catch dave@example.com. We can fix that in /etc/amavis/conf.d/50-user (which, having the highest number at the start of its name, overrides the other files, and means we can keep our edits in one place):

@local_domains_acl = ( ".example.com" );

Finally, we need to decide how many emails Amavis should process at once. It'll spin up as many daemon processes as we ask it to. You won't need many unless you're receiving a lot of emails. 2 is a common default value. We can add this in 50-user as well:

$max_servers = 2;
$max_requests = 20;

$max_servers sets the number of processes (2, as we mentioned). $max_requests gives the number of tasks that the process will run before terminating and being restarted. Strictly speaking, it's only an advisory value, but we can set Postfix to only attempt the same number (20) on a single connection, so things sync up. The default value is currently 20, but we set it explicitly here since we'll set it on Postfix's end as well and we don't want them to get out of step.

Give Amavis a restart, and you can then check if it's picked up ClamAV

/etc/init.d/amavis restart
/etc/init.d/amavis status

You should get some output like

Jan 20 14:51:22 mail.example.com amavis[21535]: Using primary internal av scanner code for ClamAV-clamd
Jan 20 14:51:22 mail.example.com amavis[21535]: Found secondary av scanner ClamAV-clamscan at /usr/bin/clamscan

The secondary scanner is a non-daemonized scanner that will run if the daemon doesn't appear to be working. You might also see here some notices that say something like "No decoder for .lz4", which means that you don't have anything installed that can unpack an lz4 compressed file, which means that Amavis can't scan it properly. If you install the relevant packages for your distro to read those files, Amavis should start scanning them for you.

The next step is to tell Postfix about Amavis. Amavis will be waiting to receive messages to process on socket 10024 on localhost (that's the default and we haven't changed it). You can see this by running:

sudo netstat -tap | grep amavisd-new

which will give the following output:

tcp        0      0 localhost:10024         0.0.0.0:*               LISTEN      1106/amavisd-new
tcp6       0      0 ::1:10024               [::]:*                  LISTEN      1106/amavisd-new

So, we need to tell Postfix to send emails to that socket so that Amavis can get SpamAssassin and ClamAV to check them. First, we need to create a new service definition in master.cf:

# Service to pass to Amavis
lmtp-amavis unix -      -       -       -       2       lmtp
  -o lmtp_data_done_timeout=1200
  -o lmtp_send_xforward_command=yes
  -o max_use=20

Here we've called our new service lmtp-amavis. It's listening on a Unix socket (we don't actually use it, we'll tell Postfix to forward anything addressed to the lmtp-amavis service to Amavis's listening TCP socket later), and has a max_proc of 2, so Postfix won't send more than 2 mails to Amavis concurrently. This should be the same as the $max_servers variable you set in the Amavis config, so that Postfix doesn't try to send too many messages at once and so that Amavis doesn't spawn a load of processes unnecessarily. Finally, we tell Postfix that this is an LMTP service. LMTP is the Local Mail Transfer Protocol, which is related to SMTP. Either would be fine, but Postfix supports multiple transactions per session on an LMTP connection, so you save a bit of overhead. Also, if the email has multiple recipients, LMTP can report a different status for each recipient (e.g. successful delivery to one, and a failed delivery to another recipient who doesn't exist). This allows Postfix to send the appropriate DSN (delivery status notifications) to the sender. With SMTP, only one status is reported back, which makes things tricky if Amavis rejects an email for one recipient but not another, since Amavis has to generate the DSN itself and pass it back.

We've set a couple of arguments with the service.

lmtp_data_done_timeout=1200 sets a time limit (in seconds, so 20 minutes) for Postfix to wait for the mail to be delivered. If the email isn't delivered within this time, Postfix will give up and report back to the sender. The default is 600 seconds (which, in the normal course of things, is already wildly pessimistic), but since we're doing more stuff now like virus scanning and spam checking, we give a bit more time.
lmtp_send_xforward_command=yes means that Amavis receives the headers in the original email for things like the original sender's IP address and the HELO command, which allows for some extra checks to see if the email looks legit.
max_use=20 corresponds to the $max_requests value for Amavis that we set above.

We also need to tell Postfix that this service is where it should send emails for checking. To do that, we need to open up main.cf and add a new line:

content_filter = lmtp-amavis:[127.0.0.1]:10024

This tells Postfix that the lmtp-amavis service is where to send mail to be filtered, and that it should get in touch with that service on localhost:10024, which is where we checked earlier that Amavis was listening.

Right now we've created a path to get emails into Amavis. However, once Amavis is done it needs to give the messages back, so we need a "reinjection" SMTP daemon to give Amavis somewhere to pass the message. This can't be the normal SMTP (port 25) or submission (587) services, because emails sent to those services will be sent to Amavis for checking. If we re-added them on those same services, the emails would go round in a loop forever. Instead, we create a new SMTP service and tell Postfix to bypass all the content filtering on that port and just deliver the email when it arrives there. We'll make sure nobody from outside can use that service, so the only email that goes there is stuff that Amavis has already checked and we're happy with.

Go back to master.cf and add the following whopper of a service:

# Resubmission service
127.0.0.1:10025 inet n    -       n       -       -     smtpd
    -o content_filter=
    -o mynetworks=127.0.0.0/8
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_delay_reject=no
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
    -o smtpd_restriction_classes=
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters,no_address_mappings
    -o local_header_rewrite_clients=
    -o smtpd_milters=
    -o local_recipient_maps=
    -o relay_recipient_maps=

As you can see, we're mostly turning things off by setting variables to empty strings. We're turning off the content_filter, various connection limits, and a load of restrictions. Emails will only arrive here as quickly as they arrive on the mailserver and get processed, so we don't really need to rate-limit them, and since the only email arriving here has been checked, we don't need to worry about a whole swathe of potential restrictions. By setting $mynetworksto 127.0.0.0/8 and setting the client and recipient restrictions to permit_mynetworks,reject we're ensuring that only local sources can send to the resubmission port. By default, Amavis will use 127.0.0.1:10025 to resubmit, so we don't need to do anything there.

We should also check that Amavis and ClamAV are correctly configured to be services that run on startup. This should have been configured when we installed the packages. We can check by running

systemctl list-units --type=service

Here you will, hopefully, see a few Amavis things, the main one being amavis.service, and two clamAV ones: clamav-daemon (the actual virus scanner) and clamav-freshclam (which updates the virus definitions, you can check its output at /var/log/clamav/freshclam.log for problems). You don't need a SpamAssassin daemon these days - Amavis is built on SpamAssassin's libraries and incorporates it automatically. If any services don't show up here, they may just not be running yet. Start up anything that's missing - I had to start clamav-daemon this way:

/etc/init.d/clamav-daemon start

After that, rerunning the previous command showed clamav-daemon as well. It's also worth checking that Amavis is fetching SpamAssassin updates, which unlike the freshclam system just uses a normal cron job. You should have a file at /etc/cron.d/amavisd-new, which should have something like this in it:

#
#  SpamAssassin maintenance for amavisd-new
#
# m h dom mon dow user  command
18 */3  * * *   amavis  test -e /usr/sbin/amavisd-new-cronjob && /usr/sbin/amavisd-new-cronjob sa-sync
24 1  * * *   amavis  test -e /usr/sbin/amavisd-new-cronjob && /usr/sbin/amavisd-new-cronjob sa-clean

That will take care of keeping SpamAssassin up to date.

At this point, we can reload Postfix and we should have virus and spam protection!

/etc/init.d/postfix reload

We can check that all seems well by talking to our new services by hand (yes, really). Open up a connection to the 10024 port Amavis is listening on:

telnet localhost 10024

You should get some output from amavisd-new, starting with a 220 status code. You can ask it what sort of services it provides with the EHLO verb:

ehlo localhost

You should get a lot of lines back, all starting with 250. If you do, all is well, type quit to exit.

Do the same thing with the resubmission port - again, you're looking for a 220 code when you connect (plus your FQDN), and lots of 250s after sending EHLO:

telnet localhost 10025
ehlo localhost

Now we need to test that the whole thing is working end to end. Obviously it would be nice to test that your virus scanner is working without actually emailing yourself a virus. Thankfully there's a standard bit of text you can send in an email that virus scanners will spot and classify as a virus for testing purposes. Send your new mailserver an email with this as the body:

X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

It's totally safe, head to https://www.eicar.org/ if you want to know more. I actually couldn't send the mail from Thunderbird, Windows checked the email for me and decided it had a virus in it...however, I was able to use my old Hotmail account to do it (thanks, Microsoft). Check the end of /var/log/mail.log, hopefully you'll see Amavis complaining with a line that starts with

Jan 23 10:56:52 servername amavis[1256]: (01256-02) Blocked INFECTED (Eicar-Test-Signature)

BAM, virus blocked, which means that Postfix, Amavis, and ClamAV are all doing their thing correctly. You can also check ClamAV's log at /var/log/clamav/clamav.log, which should also have a line about spotting the Eicar-Test-Signature. By the way, because we're passing mail to Amavis from both the SMTP service (incoming mail) and the submission service (outgoing mail), your outgoing mail is being scanned for viruses, so if you get infected Amavis should throw any infected emails in the bin before they go out.

Now for spam - as you might guess, there's a similar standard bit of text that SpamAssassin is primed to flag as UBE (Unsolicited Bulk Email, i.e. spam) as a handy test. So send yourself another email, this time with the body as this:

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

which should give you something like

Jan 23 13:39:37 servername amavis[1256]: (01256-03) Blocked SPAM

Take that, spammers. Also, yes, SpamAssassin is checking whether your outgoing messages are spam, too. This might help if your machine gets hijacked to start sending spam, although depending on the thresholds you set in SpamAssassin, it might just send stuff with the subject altered to have *****SPAM***** at the start.

That winds up Part 2. The next one will cover setting up email accounts that aren't tied to Unix user accounts by using a database-backed account system.