Why you shouldn't jump on the SPF bandwagon (2005)

Why you shouldn't jump on the SPF bandwagon Abstract:

SPF is a very broken anti-forgery technique which you should not implement. You should neither publish SPF records, nor check SPF on incoming email.

If you use SPF to reject mail, you will be causing genuine email to be rejected. There are much better alternatives which address the forgery problem without throwing the baby out with the bathwater.

OK, so what is SPF?

SPF is Sender Policy Framework , formerly known as Sender Permitted From . It makes the assumption that mail from a given email address will only ever be seen when sent directly from certain IP addresses which are associated with that address, and it gives the domain owner a way to specify which IP addresses are 'acceptable'.

So, for example, I might state that mail from the infradead.org domain will only be seen from hosts with hostnames ending in ' infradead.org ', from the IPv4 subnets 81.187.226.96/29 or 81.2.98.173/30 , or from the IPv6 subnet 2001:b80:10b::/48 . The idea would be that if you ever see a mail claiming to be from an address at infradead.org which is coming from a different host, then you should know that it's a forgery.

(That's about as far as the pretty graphical but mostly content-free official explanation goes, but if you take a moment to look closer or think about it, there's a little bit more to it than that...)

Unfortunately, this assumption is false . You do see perfectly genuine mail from my domain, from machines other than mine. This happens due to mail forwarding . People tend to change their ISP quite often, but don't want to have to tell everyone that they've changed their email address. So they have an account elsewhere, at a vanity domain or on another computer, and they forward mail from that address to whichever is their current ISP, or their employer.

SRS

There is a proposed solution to this -- the SPF advocates say that the whole world should 'upgrade' to make the initial flawed assumptions on which SPF is based come true.

Their idea is that when a server forwards a mail, it shouldn't just use the original sender's address as has been done for the last couple of decades. Instead, it should munge the sender's address as it resends the mail, faking an address at the domain which is actually responsible for the forwarding. Then, if a bounce is generated, that faked address receives the bounce and the bounce needs to be forwarded on to the original sender of the original mail.

This forwarding of bounces could be easily abused if done navely, so the idea is that we also include a timestamp to time-limit it, and a cryptographic hash which can only be generated locally. And of course you have to include the original sender's address; both the domain and the username. So when Bill Gates sends me an email to my vanity domain, for example, and my computers forward it to my current ISP, they'd actually end up rewriting the sender address to: SRS0=hh=tt=microsoft.com=bgates@infradead.org instead of leaving it intact.

This rewriting scheme they call ' SRS ', and they need it to be implemented everywhere before they can use SPF without throwing away genuine mail.

Problems with SPF

You may have managed to spot one or two potential problems with this scheme all by yourself, but some people do seem to have trouble so let's spell them out explicitly:

SPF is not compatible with today's Internet...

SRS is not common. If you publish SPF records, you are going to be asking people to throw away genuine email which you did actually send.

If you check SPF records strictly for anything larger than a toy domain with a handful of users, then you're likely to be rejecting genuine email which is sent to you or your users.

...and won't be compatible with tomorrow's either.

There is little incentive for people to deploy SRS. It's a cumbersome and convoluted workaround for a broken assumption.

Those who believe SPF to be an insanely stupid idea will strongly resist, and those who are simply uninterested in SPF have little reason to want to use it. Yet SPF requires these these uninterested third parties to upgrade too -- it's not just limited to those who are participating.

People are very slow to deploy new ideas on the Internet, and especially with respect to email, even when those ideas are good . Many people haven't even managed to deploy Extended SMTP (ESMTP) yet, even.

There will be a porcine implementation of RFC1149 before SRS is ubiquitous.

SPF is not an anti-spam technique.

Some people claim that SPF directly combats spam. It doesn't. SPF attempts to address forgery . In fact, a large amount of spam rates an SPF 'pass' result, because spammers have rapidly adopted SPF for themselves.

You still need a blacklist or other kind of trust database, to tell you which domains are trustworthy and which are not. But we already have lots of blacklists; it's just that we list the IP address instead of the domain name, to tell you which hosts are trustworthy and which are not. We don't need to break email and force everyone to upgrade to some bizarre new scheme just for that.

SPF is easily duped.

Although the uptake of SRS on forwarding sites will be low, it can still be a common spammer trick. Any spamming host can do the SRS trick to 'take responsibility' for forwarded mail. If your ISP receives mail from the address I used in the above example -- ' SRS0=hh=tt=microsoft.com=bgates@infradead.org ' -- then they have no real way of telling whether it really did come from Bill Gates via my servers, or whether it's a fake. SPF would just accept the mail, depending on my domain's reputation in the trust database. It's all about how much you trust the one server which is offering the mail -- it's not an end-to-end authentication.

So SPF is really not any better than blacklisting by IP address or HELO name. Again, the breakage it imposes is not worth it.

SRS obfuscates useful information.

The original sender address is useful information, and can be lost if an intermediate host mangles the mail by using SRS.

A sender address may be present in a blacklist, or may fail SMTP callouts. Yet an intermediate host 'takes responsibility' for the mail in question, causing it to be accepted anyway.

Sender addresses are also used for mail filtering. The sender address is the most reliable method of filtering mailing list traffic into its own folder; using Cc: and/or To: headers has both