Cascade Insights: Microsoft Hotmail Beats Google Gmail and Yahoo Mail at Blocking Spam

The Internet Patrol default featured image
Share the knowledge

Microsoft Hotmail, the world’s largest email provider, is better at blocking spam than Google Gmail and Yahoo Mail, according to a study released by the independent research firm Cascade Insights. The study only tested these companies – the so-called big three email providers – and was sponsored by Microsoft, which funded the research to combat their bad reputation for allowing loads of spam into users’ inboxes.

In recent years, Hotmail has taken definitive measures to stymie the flood of spam afflicting users. Most notably, Hotmail engineered the SmartScreen Filter, which was designed with an eye toward protecting users from the most dangerous kinds of spam, like phishing attacks. Despite the success of this technology (less than three percent of an average Hotmail inbox is spam, Microsoft claims), Hotmail’s reputation for spam-stuffed inboxes continued to haunt the company. And hence the sponsoring of the study, which seems to have largely vindicated Hotmail, or, in any case, it shows that Hotmail is a hair better than Gmail at filtering out spam, and notably better than Yahoo. (Not that spam is the only thing that users of Hotmail complain about. Every few months, it seems like Hotmail is down because of one problem or another.)

This is how the study was conducted: Cascade set up four email accounts, one for each of the three major email providers, and one for an unfiltered Web host email account, which served as a baseline. All four of these accounts were handled identically, with the test email addresses being registered at the same sites across the Internet. Some of these numerous sites were normal and safe, while others were suspicious, like sites for online pharmacies and sweepstakes. The email addresses were also posted in comments on blogs and on Facebook pages. Basically, the four email accounts were treated as if they belonged to a reckless Internet user, and, crucially, they were all used recklessly in the same way.

Cascade let email pile into these four accounts for about five weeks, and then the inboxes were analyzed to determine the amount and percentage of spam that came through. While setting up the accounts and spreading the email addresses across the Internet, Cascade kept close track of what they agreed to (like checking a box that consented to receiving emails when registering for a site) and which emails they opted in to, as they did for certain newsletters. They also unsubscribed from some of these newsletters after opting in. Of course, they monitored all these facts to determine what was spam (wholly unsolicited messages, email that continues even after unsubscribing from a list, etc.) and what was legitimate, requested email.

Composed of 48.57 percent spam, the Hotmail inbox had the least amount of unwanted email in it after the five-week period, although the Gmail inbox, filled with 48.88 percent spam, was almost identical to that of Hotmail. Yahoo Mail, on the other hand, allowed significantly more spam through, with 58.33 percent of the test account’s inbox being unwanted email. The email account that was completely unprotected was composed of 64.22 percent spam. (Obviously, these are extremely high percentages of spam for an inbox, but that is only because of the way the study was set up. The average email user’s inbox will have far less spam by percentage.)

Since the study was sponsored by Microsoft, the results of course need to be interpreted with some measure of skepticism. In May 2010, Yahoo boasted on its mail blog that according to the Fraunhofer Institute, Yahoo Mail was the best in the industry at blocking spam. Not surprisingly, Yahoo commissioned this study.

That said, the Cascade study did find that Google delivered more solicited email to inboxes than Hotmail, meaning that Google does not falsely label messages as spam as frequently as Hotmail does. So, it is not like the study confirmed (insofar as it was confirming anything) Hotmail’s superiority in every way. In fact, the study concludes by saying that “our findings […] indicate that Hotmail and Gmail are approximately equal in keeping SPAM out of the inbox.”

Regardless of which email provider is in fact the best at blocking spam, we were delighted to learn recently that several major tech companies, including the three giants of email analyzed in the Cascade study, recently signed up for DMARC, a new system designed to authenticate email senders. Hopefully there will be less spam in your inboxes soon, regardless of which company is managing that inbox.

If you feel like reading Cascade Insights’ entire report, which has plenty of details and some helpful graphics, you can find a PDF copy here:

Cascade Insights’ report on spam

Share the knowledge

4 thoughts on “Cascade Insights: Microsoft Hotmail Beats Google Gmail and Yahoo Mail at Blocking Spam

  1. I use hotmail. I set the junk filter to exclusive, this means that email only from senders on the contact list or the safe senders list goes into my inbox. I set up the junk folder to delete messages older than 10 days. There is never any junk mail in my inbox and the junk email automatically cleans itself.

  2. I agree with kodabar, my gmail account has always had far less spam than the hotmail.

  3. That just sounds like nonsense. It certainly doesn’t chime with my own experience. My gmail account perhaps lets through 1 or 2 spam items into my inbox every couple of months. In my Hotmail account about 50% of the email in the inbox are spam. That Microsoft sponsored this research is more telling than any methodology.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.