Spam Filtering on Forwarded Mail

The problem

Our mail server became clogged with messages that it is trying to deliver to yahoo.com e-mail addresses. The mail server at yahoo.com deferred them for days, refusing to accept them at all, or only accepting them after a delay of a couple of days.

Why did yahoo.com defer the messages? Although yahoo.com keeps its security practices hidden, it appears that it began to see our mail server as a spam relay. The issues appear to be:

• yahoo.com takes an unusual stance with regard to mail processing. This has caused problems at Nevis before.

• Some Nevis users forward their e-mail to yahoo.com e-mail addresses. These users receive spam, as we all do.

• Mail received at Nevis is filtered by SpamAssassin. However, mail that was forwarded to another destination was not.

• According to scans of our mail server's log files, approximately 4000 messages per day were being sent or resent by our server to yahoo.com, almost all of it spam. The yahoo.com mail server apparently concluded that we were a significant source of spam, and deferred messages from our server.

The issues

The bottom line is that there's no point in forwarding or relaying spam through our mail server. The only way I know to prevent this is to scan e-mail for spam as we receive it, which means implementing some sort of spam-rejection procedure for everyone at Nevis, not just for those who forward their mail.

The tool we use to scan messages for spam is SpamAssassin. Up until now, although every e-mail received at Nevis has been scanned by SpamAssassin, I've left it up to each user to decide whether they wanted to use it. Now I've taken away some of that choice, as described below.

The policy I am putting into place is the same policy as that used by Columbia University to filter their e-mail. In fact, I am being more generous than Columbia. However, I realize the issues involved:

• Not everyone is content with SpamAssassin's spam-finding procedure. Most people have encountered times when SpamAssassin has incorrectly labeled a message to be spam; I related one such story on the Nevis SpamAssassin page.

• SpamAssassin offers each user a way to control how messages are scanned for spam. This procedure requires that a user manually edit their ~/.spamassassin/user_prefs file. Not all the users at Nevis have the technical skill to login to the Linux cluster and edit this file.

The solutions

Here are the new policies/solutions:

• If a mail message has a SpamAssassin score higher than 10.0, it will be bounced by our mail server. The sender will receive a response that their message was rejected by SpamAssassin; the receiver will receive no indication that message was bounced. Note that Columbia University bounces messages with a score higher than 8; typical spam messages have scores of 5 or higher.

• You have control over how SpamAssassin processes your incoming message by editing your ~/.spamassassin/user_prefs file. Note that Columbia offers a way to make your spam filtering more restrictive, but offers no way to reduce or control it in detail.

  You can, for example, suppress SpamAssassin's processing of all messages that purport to be from bnl.gov by whitelisting all such e-mail. I strongly recommend against such a global whitelist; it's trivially easy for spammer to fake "From:" addresses. I recommend that you whitelist individual e-mail addresses, especially if the sender tends to compose e-mails that might be falsely marked as spam. Take a look a ~seligman/.spamassassin/user_prefs for an example of how I do it.

• Most of the e-mail forwarded by our mail server to yahoo.com are for people who haven't worked at Nevis for some time. I'm going to remove the e-mail forwarding for those addresses, and inform them separately.

• yahoo.com is now under a "three strikes and you're out" watch. This is the second time I've made changes to our mail server in response the policies adopted by yahoo.com; the first time is documented here and here. If this occurs a third time, I'm going to configure our mail server so that it will no longer forward e-mail to yahoo.com; you'll be able to send e-mail directly to such addresses, but you won't be able to put them in your ~/.forward file. If you want to forward your Nevis-related e-mail to a commercial service, please pick some other provider.