Spam Filtering on Forwarded Mail
The problem
Our mail server became clogged with messages that it is trying to
deliver to yahoo.com e-mail addresses. The mail server at
yahoo.com deferred them for days, refusing to accept them at
all, or only accepting them after a delay of a couple of days.
Why did yahoo.com defer the messages? Although
yahoo.com keeps its security practices hidden, it appears
that it began to see our mail server as a spam relay. The issues
appear to be:
- yahoo.com takes an unusual stance with regard to mail
processing. This has caused problems
at Nevis before.
- Some Nevis users forward their e-mail to yahoo.com
e-mail addresses. These users receive spam, as we all do.
- Mail received at Nevis is filtered by SpamAssassin.
However, mail that was forwarded to another destination was not.
- According to scans of our mail server's log files,
approximately 4000 messages per day were being sent or resent by our
server to yahoo.com, almost all of it spam. The
yahoo.com mail server apparently concluded that we were a
significant source of spam, and deferred messages from our server.
The issues
The bottom line is that there's no point in forwarding or relaying
spam through our mail server. The only way I know to prevent this is
to scan e-mail for spam as we receive it, which means implementing
some sort of spam-rejection procedure for everyone at Nevis, not just
for those who forward their mail.
The tool we use to scan messages for spam is SpamAssassin.
Up until now, although every e-mail received at Nevis has been scanned
by SpamAssassin, I've left it up to each user to decide whether they
wanted to use it. Now I've taken away some of that choice, as
described below.
The policy I am putting into place is the same policy as that
used by Columbia
University to filter their e-mail. In fact, I am being more
generous than Columbia. However, I realize the issues involved:
- Not everyone is content with SpamAssassin's
spam-finding procedure. Most people have encountered times when
SpamAssassin has incorrectly labeled a message to be spam; I related
one such story on the Nevis SpamAssassin
page.
- SpamAssassin offers each user a way to control
how messages are
scanned for spam. This procedure requires that a user manually edit
their ~/.spamassassin/user_prefs file. Not all the users at
Nevis have the technical skill to login to the Linux cluster and edit
this file.
The solutions
Here are the new policies/solutions:
- If a mail message has a SpamAssassin score higher than 10.0, it
will be bounced by our mail server. The sender will receive a response
that their message was rejected by SpamAssassin; the receiver will
receive no indication that message was bounced. Note that Columbia
University bounces messages with a score higher than 8; typical spam
messages have scores of 5 or higher.
- You have control
over how SpamAssassin processes your incoming message by editing your
~/.spamassassin/user_prefs file. Note that Columbia offers a way to make your
spam filtering more restrictive, but offers no way to reduce
or control it in detail.
You can, for example,
suppress SpamAssassin's processing of all messages that purport to be
from bnl.gov by whitelisting all such e-mail. I strongly
recommend against such a global whitelist; it's trivially easy for
spammer to fake "From:" addresses. I recommend that you whitelist
individual e-mail addresses, especially if the sender tends to compose
e-mails that might be falsely
marked as spam. Take a look a
~seligman/.spamassassin/user_prefs for an example of how I do it.
- Most of the e-mail forwarded by our mail server to
yahoo.com are for people who haven't worked at Nevis for some
time. I'm going to remove the e-mail forwarding for those addresses,
and inform them separately.
- yahoo.com is now under a "three strikes and you're out"
watch. This is the second time I've made changes to our mail server
in response the policies adopted by yahoo.com; the first time
is documented here
and here.
If this occurs a third time, I'm going to configure our mail server so
that it will no longer forward e-mail to yahoo.com; you'll be
able to send e-mail directly to such addresses, but you won't be able
to put them in your ~/.forward file. If you want to forward
your Nevis-related e-mail to a commercial service, please pick some other
provider.