Today I received an email from Boxbe support telling me they had finally given users the option to turn off their “coutesy notification” system.  I couldn’t be happier!  I thought I’d take this post to share about my SPAM problems, and my solution.

The Problem

GMail SPAM filtering is nice.  I may not have it forever, and don’t like to count on it, but it works very well.  Unfortunately I made the choice when I registered this domain name to set up a catch-all.  At first that was fine, but after over a year *@singpolyma.net was receiving so much SPAM, so fast, that even the GMail SPAM filter couldn’t keep up.  I began to receive over 40 SPAM (sometimes over 200) per day, sometimes all at once!  I didn’t want to disable the catch-all though… that felt like the wrong solution.

The Right Solution

I decided the right solution was whitelisting.  Since most of the people I know don’t use PGP (yet) there is no way to guarentee the sender of the messages, but from a cursory glance over my SPAM box I decided that trusting the From: header would work for 99% of today’s SPAM.

I can’t set up a forwarder from a catch-all with Dreamhost, so I set it to be delivered into a mailbox.  I then created a “dummy” Gmail account to fetch this mail via POP3.  Bonus #1, Gmail filters all this mail as it comes in, catching a huge amount of the illigitimate messages (just not enough of them).  Set Gmail to forward all email to singpolyma@boxbe.com (more on that in a bit) and delete.  Using Gmail as an email pipe/filter really.

Then Boxbe.  Boxbe gives you a you@boxbe.com email address that you can forward mail to, it checks it against a whitelist, and sends it on if it matches.  Previously, if it did not match, they would reply with a “challenge” email.  This is annoying, broken, and sometimes embarassing, so I am very pleased that they have now given people the option I wanted all along.  Disable all “courtesy notifications” and turn on the report of the queue, daily.  If I receive any mail from people not on my whitelist, I get an email from Boxbe once a day summarizing who tried to contact me.  I go and let through any legitimate new people.  Perfect.

Boxbe uses the password anti-pattern (although they’re working on fixing that, they say) to import your address book.  They have a CSV importer though.  Export from Gmail, import to Boxbe.  Set up some trusted domains (like *@uwaterloo.ca) and go.

I haven’t seen SPAM since, and have only once or twice had to go over and let through a message that got stopped.

We all hate SPAM.  We all love Akismet.  GMail is also great at killing SPAM.  Why are Akismet and GMail so great?  They have huge databases of SPAM from their many users to train filters with.

Only one problem : they’re commercial and closed.  Same old story, if they go down or evil we’re screwed.

Solution : decentralise.

The way that I’ve been thinking this could work is threefold.

First off, write a plugin for WordPress/other things that logs all SPAM in the WordPress database and allows anyone to easily access this list in standard formats.  This could hook into Akismet and other solutions to track what existing solutions mark as SPAM, as well as what users manually mark as SPAM/ham.

Then, create a site that simply lists sites that are publishing SPAM data, with links.

Third, create simple server software that either scrapes sites publishing, accepts submissions of data, or has a public API for individual SPAM submissions (like Akismet) or a combination of the above.  This server could also include filter logic that trains itself and offers a public API, or that could be other servers that rely on these ones.

The big thing is that this code all be open source so that anyone can run a server.  Each server would either scrape from all publishing sites, or publishing sites could cache a lists of operating servers to submit to.  Either way, we end up with a multiple-server environment with distributed data / load.

This is a repost of the Pranketh avoid article.

The Problem

If Pranketh’s existence proves anything, it is that email is not the safest medium around. It has always been relatively easy to send an email that says it came from someone it did not, similar to the way one can write any return address on an envelope when sending a letter. So, now that Pranketh has made this problem very obvious, how can one determine if an email is what we call ‘spoofed‘?

Some email providers and programs show warnings on messages that may be spoofed, but the problem is that detecting spoofing is more art than science. A legitimate email may be spoofed (for example, if you write Pranketh and we write you back, we are actually writing you from our GMail accounts, but it will appear as though it came from Pranketh, which, in reality, it did) or a spoofed email may not be detected (because it also spoofs whatever the automatic detection system uses).

Message Headers

First of all, you’ll want to view what we call the ‘message headers’. Some of them (From, To, Subject) are always visible. Depending on your program, different ones will usually be hidden. The option to view them all may be called ‘View Message Headers’, ‘All Message Headers’, ‘Original Message’ or something similar. Below are some screenshots for two popular email services (more will be added as time goes on) :

View Headers in GMail

View Headers in Evolution

View Headers in Eudora

View Headers in Outlook

View Headers in Outlook Express

Now that the headers are visible, there are a few key things to check for. The first is a special header added by Pranketh to all emails it sends. If this header is there, we can be sure the email was sent using Pranketh! The line will likely be near the bottom and will look like this :

X-Joke: This email is not from whom it appears to be from. It was sent from pranketh.com.

What if someone is spoofing you without using Pranketh? Thankfully, there are other things you can check. You should see if there is a Return-Path header, similar to the following :

Return-Path: <singpolyma@sunkist.dreamhost.com>

The email-address-like part of that should be similar to who it says it is from (it does not have to be an exact match, but should be similar). If it is not similar at all (i.e., the above is on an email that says it is from bill@microsoft.com) then the email may be spoofed (see the next section for more on that ‘may’).

Another header to check for is ‘mailed-by’. For example, if an email claims to be from a GMail address it may have a header like the following :

mailed-by: gmail.com

That’s pretty simple.

If none of the above is present, or if it all checks out, you may want to checked the ‘Received’ section. It will look something like the following :

Received: from smarty.dreamhost.com (d06184b1.dreamhost.com [208.97.132.177])
by spaceymail-mx3.g.dreamhost.com (Postfix) with ESMTP id 8EF98188FC7
for <feedback@pranketh.com>; Wed, 16 May 2007 16:29:36 -0700 (PDT)
Received: from sunkist.dreamhost.com (sunkist.dreamhost.com [208.97.175.14])

by smarty.dreamhost.com (Postfix) with ESMTP id 7E510EE2C4
for <feedback@pranketh.com>; Wed, 16 May 2007 16:29:36 -0700 (PDT)
Received: by sunkist.dreamhost.com (Postfix, from userid 1429516)
id 81E63402A3; Wed, 16 May 2007 16:29:36 -0700 (PDT)

Notice how there are many references to dreamhost.com. That is because this email was sent from an address that lives there (actually, it was sent by Pranketh). A GMail email will have gmail.com, google.com, or googlemail.com there instead. A Hotmail email should have hotmail.com, etc.

Maybe Spoofed

Why in the above paragraphs did we say that if any of that was true the email ‘may’ be spoofed? Well, remember, detecting spoofing is more art than science. My email addresses all live on dreamhost.com and I send most of my email through GMail, but my email addresses are all at singpolyma.net. So how can you tell the difference between an email that’s spoofed on purpose by the person that owns it, or an email that is not from who it says it is? The best way is to check emails that you know are really from them. If they are spoofed in a similar way, then the email is likely legit. If they do not normally spoof their emails, or if the spoofing looks a lot different than normal, be very suspicious.

If you are not sure an email is really from someone, write them and ask if they sent it. That way, you can be absolutely sure.

Spread the Word

A lot of people trust email every day. It is our responsibility as people who know how to detect spoofing to spread the word. Link to this article, post it on your site, email it to friends, review it, translate it, anything that you think will get the word out faster!

We only ask that you give us credit and link back to this page according to the terms of a
Creative Commons Attribution-ShareAlike license.

How to Avoid Getting Pranketh’d, Scam’ed, or Phish’ed by Pranketh is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

This was too good not to blog.  The social networking/ratings-and-recommendations site Trusted Opinion is somehow prone to SPAMers!  Billed as a way for friends to rate and recommend things for each other, the service seems good on the surface.  I was never very impressed with the insides of the site, but now I’m turned off forever.

SPAM

I received today in a ‘Private Message’ on Trusted Opinion, SPAM.  Not just SPAM, but the kind one would expect in a poorly filtered email inbox.  It’s a Nigerian money laundering scam, and real money laundering or fake, it’s one of the oldest and most well known forms of SPAM.

Pretty pathetic, if you ask me.