It's 2007 and spam is not yet under control, despite Bill Gates' predictions. Microsoft and AOL, together, have buckled under the weight of the problem. This leaves most hosts to fend for themselves, as ERI has been doing. Hopefully we've been doing well.
I thought I'd take a moment to explain how spam can be stopped and what advantages each method has. Then, I can show you what it is that ERI does to protect your inbox -- and hopefully it makes sense.
WAYS OF STOPPING SPAM -
- Content Analysis
- is the process of scanning through each inbound email, looking for spam sign. This might include certain words such as 'Viagra,' 'Cialis,' or even 'You have won.' Content analysis can be good in that it finds particular types of spam and effectively blocks those. Porn spam is especially vulnerable to content analysis. However, it also flags legitimate emails containing jokes or even close words. For example, 'cialis' is included in the word 'specialist.'
You don't want us reading your emails to be sure they're legitimate, so we are very light on content filtering here. ERI blocks emails with particular misspellings that spammers typically use to get around content filtering - like 'st0ck' (look carefully - that's a zero, not a letter O). Other than that, we don't filter on words. It's none of our business if you want Viagra emails.
- Challenge / Response Systems
- are a method of forcing the sender to verify the legitimacy of each email. You may have seen one of these; it usually reads 'Your email has been placed under quarantine. Click the link below to verify yourself as the sender of this email.' or some such.
Challenge / Response systems have a great many problems. Some folks can't or won't click on the verify link, and so the email never makes it through. Newer spam-bots typically spam a user with a return address of someone in his/her address book, so that spam is pre-cleared and passes right through the C/R system. If everyone used C/R systems, imagine the chaos - I email you; your C/R system emails a challenge; my C/R system emails a challenge back; neither of us ever see any email. Finally, C/R systems burden an already overworked email system by forcing three (or more) emails to be sent in order to deliver just one. Are you getting the idea that I don't like them? Good.
- Sender Authentication
- works in-line, as the email is being delivered. Computer A connects to ERI and knocks on the mail port. ERI does a quick double-check on the credentials of Computer A. Is it really who it says it is? Are its address records in good shape? If so, we open the mail port and allow the mail in.
Sender Authentication is an excellent means to cut down on spam, as it prevents email from many hijacked systems. ERI makes heavy use of this technology. Smart spammers, however, are now starting to fake address records. It's no longer enough.
- Reputation Analysis
- is a method of looking at the past history of a mail sender to see if it has sent spam in the past. If so, the likelihood is that it is spamming now. ERI also makes use of this technology, both globally and locally.
Globally, we rely on a list published by SpamHaus - an anti-spam service based in the UK. They publish a list of known spammers and scam artists. Each time a computer knocks on our mail port, we check its return address against SpamHaus' ROKSO list. If the computer is on that list, we deny the mail. This process works well in that many, many folks contribute to the SpamHaus list, which allows us to block the most prolific spammers.
Locally, we also rely on our own list of computers that have sent spam to ERI clients in the past. When the spam tide reaches a certain level, that computer is cut off -- it can no longer deliver email to ERI. To be safe, we are conservative with our local list. It takes a fair amount of abusive behavior to get on it; which also means it does not completely eliminate spam.
- Policy Analysis
- is the process of looking at an address on the Internet and determining whether this address should be sending email. Roadrunner, for example, has a bank of outbound email (or SMTP) servers. As a Roadrunner customer, your email goes from your PC, to Roadrunner's SMTP server, to the recipient's mail server. There it waits for the recipient to pick it up. Attempting to send directly from your PC to the recipient's mail server should result in an error or at least a non-delivery. That is how email is supposed to work.
Policy analysis involves finding where the SMTP servers are and allowing them to send, while blocking home PCs, office networks, printers and other such Internet-enabled computers. It is an excellent means of blocking email as most spammers work around SMTP servers rather than be caught spamming. However, it is an ongoing process of finding and mapping allowed and disallowed Internet space.
ERI has been using policy analysis for quite some time locally. We have a list of over 65,000 rules denoting space that is allowed or blocked from sending email based on the type of computer at the other end. Occasionally, this has blocked a legitimate email; we try and clear that up very quickly when it happens. Overall, it has been a great tool.
WHAT TO DO IF YOU'VE BEEN BLOCKED
From time to time, we get the call that so-and-so can't send us email. Your customers, vendors or colleagues are blocked from sending to you by ERI's spam rules. There is almost always a simple solution to that - provided you get the rule in question. As noted above, we have over 65,000 rules to block spam. We can't simply turn them all off. In order to find and fix the problem, we need to know which rule is causing the email to be rejected. There are two ways to do that:
First, each rejection notice contains a few code words identifying the rule. If the blocked party can relay the exact message to us (through you or by calling our office directly), then we can determine which rule is the problem and adjust it accordingly.
Second, we can watch the email attempt as it happens to determine which rule is the cause. This requires the sender to try again and notify our office so that we can investigate within four hours of the attempt. The good news is that we have a 100% success rate with this method. Just don't ask us to solve an email problem that happened two weeks ago.
THE NEW LIST
SpamHaus, publishers of the ROKSO list of unrepentant spammers, is now publishing a policy list. ERI has quickly picked it up and we will be evaluating its effectiveness over the next few weeks or so. If you have any comments - by all means, let us know!
Given the usefulness of the ROKSO list, and the amazing results we have had from our own policy list, we expect this to quickly become a powerful tool in our spam-fighting arsenal.
THE FUTURE
It's looking bleak. For many large providers (Internet companies the size of AOL, Comcast, Roadrunner, etc), a threshold was passed in 2006 - computer time spent analyzing and rejecting spam surpassed computer time spent serving customer requests. These providers will soon have to figure out better ways to resist spam ... or buy more computer power to keep up with the onslaught. It's obvious that they're not going to want to spend the money on new hardware.
WHAT YOU CAN DO
There are some simple things you can do to help stop the flood of spam:
First, use an anti-virus program such as Avast, to prevent your PC from being hacked and used to send spam. Avast is easy, downloadable and free. What more can you ask for?
Second, don't contribute to the problem by using Challenge / Response systems. They add to the noise level -- and are beginning to fail even for their own users.
Third, use an email program with good built-in filtering so you don't have to see the spam. Thunderbird is an excellent choice over Outlook. Thunderbird's spam filters are to Outlook's spam filters as Citizen Kane is to Herbie the Love Bug. It may take a month or so to train Thunderbird properly (you teach it to recognize spam by what you mark as junk), but once it's trained it really helps.
Finally, take the Boulder Pledge. Raise your right hand and repeat after me: "Under no circumstances will I ever purchase anything offered to me as the result of an unsolicited e-mail message. Nor will I forward chain letters, petitions, mass mailings, or virus warnings to large numbers of others. This is my contribution to the survival of the online community."
Good. Now maybe we can get some work done. :)