Researchers Build TCP-Based Spam Detection

itwbennett writes "In a presentation at the Usenix LISA conference in Boston, researchers from the Naval Academy showed that signal analysis of factors such as timing, packet reordering, congestion and flow control can reveal the work of a spam-spewing botnet. The work 'advanced both the science of spam fighting and ... worked through all the engineering challenges of getting these techniques built into the most popular open-source spam filter,' said MIT computer science research affiliate Steve Bauer, who was not involved with the work. 'So this is both a clever bit of research and genuinely practical contribution to the persistent problem of fighting spam.'"

Comments

Why do we keep doing this?

[damn_registrars]

People are looking at the wrong end of the problem with much of their efforts - and this is just another example of that. You cannot solve spam with filtering, detection, or legislative actions. We've seen time and time again that those are just time and money-sucking stopgap measures that ignore the reality of the situation.

We won't see a real solution to the spam epidemic until people acknowledge the simple truth that spam is an economic problem. There is still a lot of money to be made by sending out spam, with very little expense for the spammer. The profit margin is high enough that it is well worth their while to find various ways around filters and any other silly mechanisms we throw at them.

If you want to make an actual difference in the fight against spam, you need to approach the economic motivations behind it. If you stop of the flow of money to the spammers, you will stop the spam as well. Because no matter how much some people may want to believe otherwise, spam isn't sent just to piss you off and ruin your day. Spam is sent out because spammers are paid to do so. If they don't get paid, they won't send spam, it is as simple as that. Any other kind of countermeasure only prolongs the fight and throws more money in the wrong direction.

Re:Why do we keep doing this?

[Tom]

The economic side has been tackled as well, and it turns out that it is not easier than the technological side. More importantly: It involves politics, and politics move slowly on all problems of the commons (i.e. low impact on many people).

Re:Why do we keep doing this?

[wbr1]

For the same reason we have security theater.
For the same reason we have a 'War on Drugs'.

We seem to be blind to the fact (as a society or a government), that you cannot legislate or regulate a cure to a problem. People will always do what seems in their best interest, be it recreationally, economically, or otherwise.

Very little our government does actually address the core issue, it just places band-aids on top of it. This, I think at least partly because a democracy is a system of compromise and once you have compromised the strength of many solutions is sapped by that compromise. This is not to say that it never works, but it is degrading quickly. Creating stop-gap solutions and band-aids helps those in power feel like they have made a difference, and for the most part the willfully uninformed public follows and agrees.

As a case in point take the drug war. It would be unfeasible to say shoot all smugglers and dealers. It is also impossible (in our current society) to say legalize all drugs. Instead we have a multibillion boondoggle of a system to try to stop and regulate illegal drugs, and it has never worked. In addtion, even though it is obvious that it doesn't work, there are those who benefit from it's existence and will push to keep it even if it is a failure.

There will always be those who find it better to game the system than to stay within it.
There will always be a disenfranchised group who feels they have to act differently than the norms for the interest.
There will always be those who feel they have the tight to abuse or prey on others.

Re:Why do we keep doing this?

[Halo1]

The same can be said about pickpocketing, burglary and almost any other kind of crime. As long as technical measures can help with partially or temporarily alleviating the problems without causing disproportional side effects or requiring disproportionately large investments (i.e., not TSA nonsense vs terrorism, but more like door locks vs breaking and entering), I don't see what the problem is with developing and deploying them.

Re:Why do we keep doing this?

[kelemvor4]

Anonymity is not a feature inherent in e-mail.

Re:Why do we keep doing this?

[Mister Whirly]

Yes, paying for all email. I can't see any drawbacks to that solution.

How about instead of elecrtronic mail, we devise a system where people write letters on physical paper and then we deliver those letters to the recipients. We could charge a nominal fee for the delivery, and that should end all "junk mail", right?

Re:Why do we keep doing this?

[wkcole]

Here's an idea - recipient's SMTP server refuses e-mails unless they get 0.01 cents with it.

Don't bother trying to patent that idea. It has been proposed and even tried many times.

One problem with it is simply that there is no reliable mechanism in place to identify the responsible sender of every piece of email. Internet email is not a single system, but rather a loosely confederated mob of independently operated systems that mostly use a common set of protocols. Most email these days is spam, sent mostly by hijacked machines, of which most is rejected easily by most receiving systems. The bulk of spam that makes it to user inboxes is either being sent in ways that are intentionally deceptive and often using stolen resources or is arguably not really spam because it is pursuant to some formally (if ignorantly) accepted agreement to be sent mail. Neither of those is easily addressed by making rules for people to follow. The first set are not going to follow any new rules and the latter are working within the letter of the existing rules.

Re:Why do we keep doing this?

[damn_registrars]

How do we know this? Because we can observe (and we have observed) that they continue spamming even when there's obviously no profit in it, nor any realistic hope of any profit in the future.

That is simply not true. There is plenty of money to be made in spam, and it is the motivating force behind it. The spammers that make the news when they get caught (almost always on other offenses) are especially wealthy relative to their home countries. Furthermore, the total investment for a spammer is minimal; they really just need to be able to talk a good game and get some time on a botnet to be able to make money fast. As we've seen, each time a spammer is thrown in jail or murdered , the spam volume at best remains the same (more often, it increases) because it is profitable.

Your very notion of spammers being inherent sociopaths simply makes no sense. If they just want to aggravate people electronically, they could do it by trolling discussion forums and not have to worry about what side of the law they are on. They are not all mentally ill, they are all just looking to make a buck. And many of them - have you ever looked at the lists on spamhaus? I'm guessing no - are from former second-world or current first-world countries where economic opportunities are scarce.

Re:Why do we keep doing this?

[Tom]

In a way, though, it is. There are actually fewer actions that need to be taken from the economic side than from the technological side; indeed economic actions can have very measurable and lasting effects in a short amount of time while technological actions are generally worthless.

Do you say that as an economist or as a technician? Because I would take a bet that the other side would say the same thing, only in reverse.

You may have misread me on that matter. Economic solutions are not inherently political, even though politics is inherently tied to economics. However, the companies who are on the financial take in the matter can be influenced without the necessity of legislative action.

If it were that simple, someone would have done it by now, don't you think? If it is just that nobody has done it, then why don't you?

Won't work

[sakdoctor]

Even if the spam click-though rate is 0.0%, there are still enough suckers born every minute to buy the service of spammers.

Skip the ITWorld article

[wkcole]

I'm sure 'itwbennett' would rather everyone go to his employer's website to read that article, but it is clearly not written (or edited) by anyone who has any basic clues about spam-fighting. Just reading the subtitle makes me cringe for the unfortunate "journalists" lassoed into writing it, as it was clearly done by spam neophytes in a desperate scramble for click-scrounging content. The article is vaguely about a paper presented almost a year ago at LISA '11. There are links to an abstract and the original paper at the LISA '11 site: http://www.usenix.org/events/lisa11/tech/

The general space of sniffing out spam by looking at TCP characteristics has been mined for years usefully with Symantec and MailChannels both offering proprietary tools that use such techniques and some open DNSBL's using TCP sniffing to identify sources, but it would be incorrect to believe that any one methodology will ever be a magical silver bullet against spam.

Re:Looks like a copy of someone else's work...

[MightyMartian]

Postfix has had throttling for several years now, based on the same basic concepts. I use Postfix with greylisting and to be honest, my Spamassassin and ClamAV filters rarely get hit. Since at least big spam attacks are by bots, and bots are primarily designed to just shove as much through as possible, greylisting alone does a spectacular job of killing them, though sometimes people get pissed when messages take a while to get to them from a recipient the first time.

Please stop

[WaffleMonster]

I've always wondered how seemingly smart people can act so stupidly totally oblivious to the repercussions of their actions.

What happens when a busy computer that would cause it to naturally act in a similiar matter as a botnet zombie sends an email and that message is then flagged as spam?

Spammers are no fools or dinosaurs. They will simply adjust their spamming rate in zombie client below the threshold needed to induce effects needed to trigger the detection scheme.

End result as always is the same:

It won't stop anyone from spamming

It WILL make SMTP based Email even more unreliable than it currently is.

Re:Please stop

[DamonHD]

This rather assumes that every MTA will have the same threshold. It is not necessary (or helpful) to have a security monoculture.

A very simple first defence against such rate tuning is to randomly vary thresholds substantially between systems and from time to time.

Rgds

Damon