Personal Filtering Tools and RBLs

One thing that is certain every Thanksgiving and Christmas season is that mail traffic will significantly increase both in terms of numbers of messages and the average size of messages. Part of this traffic increase is due to folks sending seasonal greetings and e-cards but a large part of the increase is due to increases in spam and spyware/malware messages (phishing, etc.)

What also increases is the number of "false positives", messages rejected or marked as spam that should have been accepted as valid. For instance, the e-card sent to you by Grandma that keeps getting bounced as spam or spyware.

Many of these false positives come not from commercial mail servers equipped with filtering technology but from the millions of filtering products in use on personal computers. Sadly, most of these products are not setup correctly which in term increases the number of false positives.

Installing a product such as MailWasher is usually very easy and clean; developers put a lot of thought into how to make their products "load and go". The problems begin post-install, the time when users are able to control the severity of the filtering. This is when you can add blocks for annoying messages from a former friend or instruct the filter product to not block messages from your favorite mail list.

Many of these products make use of one or more of the many Remote Blackhole Listing services, (RBL). In many cases the user is able to add or remove RBLs. Unfortunately, most users do not have the background to make these decisions based upon a thorough review of how a particular RBL functions. The end result is usually frustrated users and a flood of angry emails to the ISP/ASP's support folks.

The following material is the result of one of these 'angry messages'. The content has been cleaned up a bit to protect the identity of the sender and to fix a few spelling errors.


Hello XXXXX,

You want to be careful about relying upon the RBL lists. It is not unusual for a domain or ip to show up as blocked for a day or two and then clear up. Even the major services such as AOL have been listed by one RBL or another at various times.

[As of two minutes ago. On openrbl checking for 195.92.246.182 on 38 blacklists showed the ip is positively whitelisted on 4 lists, positive blacklist on two (spamcop and spambag), neutral whitelist on one and neutral blacklist on 29 lists.]

We each have to decide for ourselves what RBL lists to trust but for what it is worth, we stopped using spamcop with our commercial mail servers due to far too many false positives. After a lot of testing, tweaking and client feedback, we have found a set of RBLs that seem to work fairly well: sbl.spamhaus.org, relays.ordb.org, and
relays.mail-abuse.org. These services place more importance on whether or not a domain mail server is an open relay or not since almost all spam flows through open relays.

It is also important to remember that each RBL service has a different set of criteria for listing someone. Some of these lists are known to be very lenient, others are known to overly sensitive. Some go through a very thorough investigation, others merely react to a report from a disgruntled reporter. Some accept and act upon content complaints, others only react to open relays.

The bottom line is that there are a ton of RBL lists, each with it's own philosophy on how to filter and it is important that you understand their processes and criteria if you are going to use them.

Our advice to our email clients is to not do any RBL checking on their local machine (we do that at the frontend as well as spam assassin and bayesian filtering). If you want to filter, which is not a bad thing since no frontend filtering system is 100% foolproof, do so based upon content. For instance, the Thunderbird email client has an excellent Junk mail filter component that is completely within the control of the user. You decide what is junk and what isn't. I don't use Outlook but my understanding is that it has something similar.

If it is any consolation, over the last 3.5 years, we have seen Smartgroups listed and delisted so many times that I stopped counting. ;>

Rich
Dogsoldier.com, LLC