Gmail’s Spam Filter Isn’t So Hot, But We Can Get You to Zimbabwe Cheap!   4/18/2004 - 1,484 views, 2 Comments

In addition to openly soliciting email to the account, and even posting the address on the web (shedevil@gmail.com), I’ve forwarded all of my email coming to my other accounts to the Gmail account. I wanted to give Gmail a good workout. More specifically, I was very interested in its spam filtering capabilities (well, I would be, wouldn’t I?)

At the end of nearly a week, I can say that my first impressions hold: Gmail’s spam filtering performance, to put it politely, sucks.

Out of a total of 259 pieces of email (in part so low because of the intervening weekend, and in part because I had to turn forwarding off for most of a day because my server and the Gmail server got into an infinite loop argument over whether Gmail should continue to bounce its own bounce messages into perpetuity, my server happily playing the me so forwarding game)..anyways, out of a total of 259 pieces of email, Gmail tagged exactly 11 of them as spam, of which only 4 (fewer than half) were actually spam. That’s 7 pieces of legitimate email shunted to the spam folder - which might be a whole lot more acceptable if Gmail had also managed to catch more than 4 of the *62* pieces of spam represented in that corpus of email. And I’m not talking borderline spam here - I’m talking about serious Logos and Nigerians and Viagra, oh my, spam.

So, let’s see - Gmail’s spam filters shunted 7 pieces of legitimate email to my spam folder, while letting 58 pieces of serious, gnarly spam into my inbox.

Not exactly a stellar performance.

Justin Mason, at taint.org, did a far more substantial test and analysis in his write-up of the Great Gmail/SpamAssassin Spam-Off (ok, I called it that, he actually calls it “Some stats on GMail’s spam filter”). In his test, Gmail demonstrated a false negative (spam in the inbox) rate of greater than 31%.

But still, on the bright side, Gmail’s Adsense targetting is working really well! Why just today, in an email from Mrs. Nwakama Ani, the wife of the late James Ani, a farmer in ZImbabwe, asking me to please help her to export $50million dollars which her late husband amassed, Gmail’s Adsense very thoughtfully offered me “Cheap airline tickets from the USA to Zimbabwe”. You know, just in case I want to go over there and help her personally.