Slashdot Mirror
Microsoft Researchers on Stopping Spam
TheBackBencher writes "Scientific American today has a very interesting article about "Stopping
Spam" by Joshua
Goodman, David
Hackerman and Robert Rounthwaite from Microsoft Research. They talk about different types of spam -- spam with emails, spam on IMs, spamlinks
on web pages and image based spam. They mention different techniques for
spam filtering mainly fingerprinting matching techniques, n grams model,
naive bayesian approach, optical character recognition, challenge/response systems and Human Interacted Proofs (HIP) in a very lucid style. They however do not mention fingerprinting approach of using Nilsimsa Hash to
tackle addition of random words by spammers in emails or hypertextus interruptus technique used
by spammers of splitting words using HTML comments, pairs of zero width tags,
or bogus tags. Also, Spam-Research is reporting the
SplitFit
Technique that Spammers are using to fool Yahoo! Mail SpamGuard."
If it was developed it can be reversed engineered. Sorry to say but spam is here to stay unless of course someday the internet becomes regulated somehow.
Of course, one 200MB update from Microsoft would kill this idea. Or how about a 500MB game demo download? Thats legitimately free. Or better yet, what if I need to download a linux distro or a television episode?
I would hate to have to explain all my actions to my ISP. Espically with the way media is driving the internet nowadays. 200MB is way too small of a limit.
Now, you can monitor how many e-mails are sent by a host. That would be a better way. At least there could be a filter on the "to:" line. If that list includes over say, 1000+ users, consistantly, then at least there could be some flags raised.
give spammers a 9 year prison sentence.
That'd probably be the best thing M$ could do to help reduce spam.
I shudder to think on what you mean by a "bounce" feature. Most likely sending a "bounce" reply to the forged sender address? That's part of the problem, not the solution.
One line blog. I hear that they're called Twitters now.
Interesting idea, however invalid address responses are sent within 5 minutes of the original mail. If the response is sent over a day after the original mail is sent, the spammer could just discard it.
If we respond instantly to all email with invalid address mails, it will be overused and spammers will ignore ALL of them. This is much like antibiotics, we use them too much, the bacteria accommodates for it, and antibiotics become obsolete.
So far, bayesion filtering and/or whitelisting email are the only good systems of blocking spam at the moment.
1) It is a form of communication
all email is communication
2) The communication is unwanted
"wanted" is a subjective property of the recipient - the computer has no programmable decision procedure for wantedness.
3) The source of the communication is hidden
There may be some system of authenticating sender ID, and will be as easy as getting ppl to use pk encryption.
4) In recieving the communication, you use your bandwith or incur a cost
again a property of all emaiil.
try { do() || do_not(); } catch (JediException err) { yoda(err); }
I don't undertsand this. On one hand, you have the police saying they can't track spammers. Spammers use drones, they remain hidden, they hide their tracks. On the other hand, if you unsubscribe, they know your email is a real one, and you get more spam. That tells me whoever runs the unsubscribe service is in cahoots with the spammer and is just as guilty. They have to know where to send their lists? Just track them as part of the war on spam.
Rosco: "If brains were gunpowder, Enos couldn't blow his nose."
Here's a more interesting idea...
Authenticate SMTP with public key signing. -- Then use a trust network to only accept email from trusted companies.
Why it won't work:
It involves effort and cost.
Baah, the internet should be unregulated, if they can get rid of SPAM then whats to stop them getting rid of porn, anti-government information etc. There's a road we all want to go down.
Don't buy it and Get over it(tm).
The article barely mentions economics, and only in terms of the real costs of email--which only shows how much room there is for a real economic model with real business, real email, and *NO* spam.
I really wish one of the major email players would offer an option for prepaid email. That would be an absolutely spam-proof system. It doesn't matter if the postage is two cents, the spammers can't afford it. Two cents against 50,000,000 spams turns out to be *REAL* money. Any email via that address would be at least some kind of real thing.
Freedom = (Meaningful - Coerced) Choice != (Speech | Beer^2), and sad sock puppets' bad mods avail them naught.
First, I guess you didn't see the guy in VA who just got something like 9 years in jail.
That said, spam doesn't obey jurisdictional boundaries. Any single country can only solve a small part of the problem, and any spam incident often involves over 3 jusrisdictions that may be in separate countries (sender, spambot, recipient, etc). That's a logistical nightmare that isn't soluble outside of a dream world.
Also, force all ISP's to monitor how much bandwith a source has. If you get too much usage per day, say 200 megabytes or more, then that person has to explain why they need that much bandwith. If someone gets the RIAA on board, with their lobbyists, that should pass very quickly.
That's fantastic. Trade a bad problem for one that's much worse. Get the RIAA to legitimize their practices by using a guise of stopping spam? Let's not.
Also, force all email to have some element which identifies the source. Not just a header that can be forged, but something that can't be hacked.
Now by force, what do you do if they don't? Enforement issues again here.
Ultimately, legislative solutions for spam DO NOT and CAN NOT work for much but a small part of the problem. It's satistfying when some moron is clumsy enough to get caught (as with the guy in VA), but mostly these days the spammers aren't that stupid. Technological solutions work far better.
Is there really such a thing as a solution to spam? For every new technique that is developed, the spammers will find a way to circumvent it. Spam is a multi-million dollar business. I'd go so far as to say that it's a science. At least, the spammers seem to have it down to a science.
Trying to find a solution to spam is an idea in the eyes of experts and analysts. But to spammers, it's a road block that they must work around to stay in business.
Spamming techniques will no doubt end up as signatures in spam filters that are not unlike those signatures used by IDS and virus scanners. The experts don't seem to understand that if there's a will, there's a way. And the spam will just keep coming in another form or by some other technique. All that can be done is to keep up with changing techniques and patterns and treat spam for what is truly is -- an attack vector.
Right now spam is usually filtered using a brownian model. As a result, spammers have begun structuring their emails so as to target brownian models. How many spams have you gotten lately with the subject line ending in confiscate ok wallop yls oblivion?
If we move to filtering spam using markov models, spammers will begin structuring their emails so as to target markov models. Look forward to all your spams ending in 500-word blocks of text from a copy of MegaHAL trained on old grandmothers' email boxes.
Irritable, left-wing and possibly humorous bumper stickers and t-shirts
In the past the FBI has already caught people by simply buying what they were selling and then finding the person who cached the check.
Of course the FBI can't arrest people in the lawless places of the world like croatia and hungry so those government will need to shed their corruption.
In other words I don't think your scheme will work because so much of the world is out of the reach of law enforcement.
evil is as evil does
Interesting idea, however invalid address responses are sent within 5 minutes of the original mail. If the response is sent over a day after the original mail is sent, the spammer could just discard it.
The thing is, I don't belive spammers ever remove an address due to an error. I had a domain that received a ton of spam, and that domain expired. Two years later (fighting with Network Solutions) I got the domain back, and immediately started receiving a ton of spam. Two years of spammers sending spam to invalid addresses (no DNS on the domain) and they still continued.
Why?
Simple: the spammers don't receive bounce messages, and the spam-servers (which could be static servers, or compromised zombie machines) don't provide accurate return information. Much like how telemarketers often show invalid or "Unknown" caller-ID info. It costs nearly nothing to send a spam message to an address, whether that address is valid or not. It costs much more to weed out invalid or unreachable addresses from your list by intercepting bounce messages etc.
And spammers don't give a shit. Most of the time, they are using someone else's machine (a zombie'd Windows box, or an open relay) so they don't need to care. So this trick simply doesn't work. It's cheaper to just continue sending to invalid addresses. Not to mention, many newbie spammers get their lists from less-than-legit sources who are selling large lists; they don't care (and are usually fully aware) that many of the addresses they are selling are bogus or no longer valid...
In short, simple tricks like this don't work, when dealing with an "industry" that doesn't give a shit...
NGWave - Fast Sound Editor for Windows
You haven't RTFA it seems.
That garbled text is ungarbled by certain software (i.e. outlook). That's because there are invisible chars in there that activate the "right to left" mode.
Example:
De*ra* B*lcra*ays M*bme*er
translates to:
Dear Barclays Member
(I tried to copy the text I got in Yahoo, and paste it in MSN messenger. Amazingly, the text was "ungarbled". That's when I realized how tricky spammers were)
SPAM software could simply detect left-to-right characters in such text, and ipso-facto label it as spam. Unless of course, you're reading hebrew. Which is obviously NOT the case.