My Adventures in Spam Blocking

I hate spam.  I detest it.  And I'm not talking about the scrumptious processed meat product.  I think you all know the spam I'm referring to—the kind peddling cheap viagra, phony rolexes, and desperate but gorgeous Russian girls pining for your love.

Most people I know pack up and leave an email address after it starts getting loads of spam, but being a dedicated IT nerd I didn't want to throw in the towel that easily.  I made it my life's mission to stop all spam from getting into my inbox.  Well, maybe not my life's mission, but my primary home IT project for the next few months or so.  This was my mission, and this is my story.  Maybe you'll see this in theaters next summer.

My Own Mail Server
If I was going to totally block all spam from my inbox, I needed absolute control over my email infrastructure.  Free email like Gmail or web hosts just weren't going to provide the tools, configuration options, or software I required.  Thus, I migrated my email account from my web host to my very own email server built on Ubuntu server and Zimbra.

Out of the box Zimbra includes impressive spam & virus filtering—ClamAV, Spam Assassin, spam training, and DNS blacklisting.  It was a handy arsenal that I appreciated since it's far more tedious and time-consuming to install and configure all those packages manually.  Zimbra was the most comprehensive free email server I found.  Attractive also was Citadel, but Zimbra's webmail interface captured my heart.  I played around with both for a bit before deciding on Zimbra.

Know Thine Enemy
Next I studied my enemy.  What countries was this crap coming from?  Were the source IP addresses on existing blacklists?  If so, which ones?  When spam would get through, I would examine the headers and find the source IP, gathering useful info from a service like whatismyipaddress.com.

I learned 2 main things—most spamming mail servers were blacklisted by at least 1 blacklisting service; and most spam was coming from countries that I couldn't give a flying rat's ass about.

Blacklists
DNS blacklists (aka blackhole lists, DNSBLs, etc.) are online lists of mail servers that have a bad reputation for one reason or another: They relay spam, they aren't secure from spam bots and other threats, they kill kittens, etc.  There are myriads of lists maintained by myriads of organizations or individuals.  Fortunately, personal use of the lists is most often free.  Most modern email servers can use multiple lists.  Whenever inbound mail comes, the server will check if the sending server is on a blacklist.  If so, the mail is refused.

During my research phase, 2 lists barked at me:
* zen.spamhaus.org
* b.barracudacentral.org

Spam sources were almost always on 1 of these 2 lists, so I configured Zimbra to use them both.  Barracuda's list requires registration, but both are free for personal use.

As I expected it didn't eliminate spam 100%, but it made a massive difference.  I now had far less spam getting through.

DNS and SMTP Protocol Checks
Zimbra and its underlying Postfix architecture includes extremely effective DNS and SMTP protocol checks that can be enabled simply by checking them on.  These checks block mail from improperly configured mail servers.  For example, if the sending mail server does not have a proper resolvable hostname, then the mail is rejected.  Since few spamming mail servers are "properly" configured (many are botnet zombies and virus-infected PCs), I have found these checks to be an indispensable arsenal against spamming servers.

These are the checks that I see in Zimbra:

Protocol checks
Hostname in greeting violates RFC (reject_invalid_helo_hostname)   
Client must greet with a fully qualified hostname (reject_non_fqdn_helo_hostname)   
Sender address must be fully qualified (reject_non_fqdn_sender)   
   
DNS checks
Client's IP address (reject_unknown_client_hostname)   
Hostname in greeting (reject_unknown_reverse_client_hostname)   
Sender's domain (reject_unknown_sender_domain)   
Client must greet with a resolving hostname (reject_unknown_helo_hostname)

After some research and testing, I turned on all the protocol checks, along with the "reject_unknown_reverse_client_hostname" and "reject_unknown_sender_domain" DNS checks.  I did not enable the other 2 DNS checks because they were reported by others as causing a high number of false positives.  It's so-far-so-good with my current configuration--I rarely experience false positives.

Since these checks are so effective and widely-implemented, IT admins realize they'd better properly configure mail servers if outbound mail is going to work.  If my server does reject legitimate mail, the sender receives a bounce-back stating that something is wrong with their mail server configuration, and I see the rejection in my mail logs.  Worst case, I turn off the checks temporarily to receive the mail.  The few times this has happened, the sending company appreciated my free IT advice.  Yes, I am that dorky.

SPF Check
SPF stands for "sender policy framework."  So what does that mean?  It means domain owners can add a DNS record that states what servers are authorized to send out their mail.  Email is not secure, so it's susceptible to spoofing.  If you've ever received a spam email from yourself or a phish email from DHL that looks like it was from the DHL domain, then you know what spoofing is.

SPF is designed to prevent spoofing, but receiving servers have to be configured to check SPF DNS records.  Here's how I enabled SPF checks in Zimbra: Setting Up SPF on Zimbra Running on Ubuntu

Country Blocking
My next goal was to implement country blocking.  Every country in the world is assigned certain stocks of IP addresses, which makes it possible to know where mail traffic is coming from geographically.  Since I had my own mail infrastructure now, I could in theory block email from certain countries.  As this kind of block is at the server or network (not individual mailbox) level, for fairly obvious reasons email providers can't offer this degree of tuning.

Zimbra didn't offer this feature, so I looked into doing it at the point of entry—the firewall.  Untangle was my firewall of choice at the time, but it too didn't offer this feature.  I found an awesome free open source firewall project that did—PFSense.

Compared to Untangle, PFSense is far more complex.  But I wouldn't think of complaining about the added complexity considering PFSense offers every bell and whistle that Untangle charges money for.  Once I got my head around PFSense, I realized how insanely powerful and extensible it is.  For example, PFSense has an add-on package called PFBlocker that allows you to block or accept traffic from certain continents or countries.  I had to wipe the drool from my lips.  More about my open source UTM firewall research can be seen here: The Hunt For the Ultimate Free Open Source Firewall Distro

I don't know anyone in Poland, Brazil, Russia, or Romania, so I flat-out blocked all inbound mail from those countries.  Several more countries were added later.  Sure, it's incredibly aggressive, but incredibly effective at the same time.  I was slicing heads off of snakes one country at a time.  I also setup a custom IP blocklist on the firewall and added spammer IPs and even entire IP ranges that kept spamming my server.

One final note about country blocking is that it can be accomplished by using a blacklist.  I found a few sites offering not spam blacklists, but geographical IP lists that can be used the same way.  For example, configuring a blacklist look-up to cn.countries.nerd.dk would block all mail from China.  I considered this approach, but realized it could become tedious if I wanted to block a large number of countries and/or wanted to block more than just mail traffic.

One Remaining Spam Source
I now very rarely receive spam, and most of the spam that still got through came from 1 source—fake Microsoft Hotmail / Outlook accounts.  I figure I'll be waiting forever for Microsoft to get their act together, so I flat-out blocked all mail from the hotmail and outlook domains, then whitelisted the few friends' addresses that use that service.  I'm tempted to do the same with Yahoo.  I'm no Google fan-boy, but when it comes to spam, Google seems to really have their act together.  I very rarely see spam messages relayed though Google's servers.  I get spam from Hotmail and see Yahoo email accounts compromised all the time, so if I were a “normal, non-nerd” email consumer, I'd probably go with Gmail.

Final Notes
I've been happily running my own mail server that blocks significant portions of the earth from emailing me for a few years now.  Looking at how aggressive my approach is, you'd assume I worry about accidentally blocking legitimate email.  Not really, and here's why:
- Friends and family have multiple ways to contact me.  There are so many communication alternatives these days that email isn't that essential for personal communication.  If I block a friend's email, they could simply Facebook, Skype, Twitter, LINE, etc. me.  In fact, most friends and family prefer those methods anyways.  Email is kind of dying out, especially in the personal realm.
- Important emails from financial institutions or Google or Apple or Amazon always get through.  Their servers are properly configured and have sparkling reputations.  It's almost always smaller companies running their own mail servers that get blocked.  To the best of my knowledge, I've never blocked anything truly critical.  Knock on wood.
- It's my own mail server, and I'm the only user on it.  I have no one to answer to except myself, and I'm a patient and understanding customer.  Most importantly I appreciate that I've had the same email address for well over a decade and receive maybe 1 spam email a month (which is promptly pissed-on and exploded).  I don't think many people can say that.  If you can say that, then let's go out for drinks sometime!