Slashdot Mirror
Microsoft Researchers on Stopping Spam
TheBackBencher writes "Scientific American today has a very interesting article about "Stopping
Spam" by Joshua
Goodman, David
Hackerman and Robert Rounthwaite from Microsoft Research. They talk about different types of spam -- spam with emails, spam on IMs, spamlinks
on web pages and image based spam. They mention different techniques for
spam filtering mainly fingerprinting matching techniques, n grams model,
naive bayesian approach, optical character recognition, challenge/response systems and Human Interacted Proofs (HIP) in a very lucid style. They however do not mention fingerprinting approach of using Nilsimsa Hash to
tackle addition of random words by spammers in emails or hypertextus interruptus technique used
by spammers of splitting words using HTML comments, pairs of zero width tags,
or bogus tags. Also, Spam-Research is reporting the
SplitFit
Technique that Spammers are using to fool Yahoo! Mail SpamGuard."
I have an idea for stopping spam. How about sent mail is not flagged as recieved until it is viewed by the user. Once it is viewed, it can be flagged as good or flagged as spam. If the mail is flagged as spam, the mail could be sent back to the orignal host as unreturned mail.
So for every mail that is sent out, once flagged as spam, could be sent back. Of course, normal spam filters would also flag the data as spam. If the e-mail is returned to the host, maybe they will remove the e-mail from thier list. Otherwise they will just recieve a bunch of spam as well as sending it. It could clog up bandwith pretty good.
However this probabally wouldn't work if they somehow spoofed the sent address.
If Microsoft email clients had a "bounce" feature spam mail wouldn't be such a problem. Microsoft should take a lesson from KMail. Ha!
No, Spam is easy to define, it is any unwanted emails. Name elements that make spam:
1) It is a form of communication
2) The communication is unwanted
3) The source of the communication is hidden
4) In recieving the communication, you use your bandwith or incur a cost
Rosco: "If brains were gunpowder, Enos couldn't blow his nose."
Dera Blcraays Mbmeer, Thsi eamil was stne by the Barclays serevr to vreify yuor emial adsserd. You mtsu competel thsi pssecor by ccilking on the likn bewol and entireng in the smlal wiodnw yoru Braclays Membership nrebmu, passcedo and meelbarom word. Tsih is doen for yruo proteoitcn - buacese semo of our mrebmes no lonegr haev assecc to theri emlia adserdses and we muts virefy it. To vyfire yruo eiaml arddess and accses yruo bnak anuocct , cilck on the lnik bolew:"
That email is extremely difficult to filter out because the only 'real' words are no, of, our, and, etc. Simple words that occur so many times in legitimate emails that most spam filters practically ignore them. But I have to wonder.. who would actualy 'cilck on the lnik bolew' anyway? I hate to use the term 'you get what you deserve', but if you are naive enough to click the link, then the problem isn't your spam filter, it's you.
Plus, with the postal service, there are 1000's of laws in place. If I send you an offer through the mail designed to rip you off, that is a federal offense. You can't use the US Postal Service for illegal activities, if you do you get caught.
Remember the movie The Firm? They did not convict the lawyers for tax evasion or any other crime. They convicted them for mail fraud. And if you let the worst spammers know that each and every time they send a message that is spam, each instance will incur a penalty, that might stop them.
Rosco: "If brains were gunpowder, Enos couldn't blow his nose."
Here's my solution to the greater unwanted communication Anti-spam paper submitted to Conference on Email and Anti-Spam
Error: Id10t detected
Legally, this is promising. First, there's no free speech issue. Second, in most jurisdictions, it's illegal to operate an anonymous business. So most spammers are criminals. Third, laundering transactions through intermediaries is usually a crime, too.
The problem for law enforcement is that following the money is difficult. Additional technical support for that would be a big help.
A good starting point would be to get a credit card issuing bank to cooperate in a scheme where, when one of their credit cards is used, full transaction details, including the payee's full identity, are immediately returned to the cardholder, using encrypted E-mail or some other secure means. That would make "following the money" much easier. This only requires one cooperating bank. That bank's credit cards might become popular with heavy Internet users. Especially if this works for prepaid credit cards, so you can find out who's behind a web site by using some disposable credit card.
The next step is to crack down on "credit card intermediaries". Non-bank credit card intermediaries that handle spammer transactions should be stuck with the legal liability of the spammer. Legally, they're the "merchant". They shouldn't be allowed to pass the buck to some other party. This will make "cheap merchant accounts" harder to get, which is probably a good thing.
Why not use a hidden markov model to filter spam that use random digits as filler?
A very basic filter will work this way:
Train a network of say, 30 to 40 units, with any english text. The training text doesn't just have to be limited to letters and numbers, it can include other ascii characters as well, because the hidden markov model will create distributions for them as well.
Now, for each new email that comes in, grab random chunks of text (maybe random 30-character strings) and see how probable the text would be in this hidden markov model. If it turns out not very likely, then scrap it.
Any thoughts?
Yeah, that won't cause much problem for bayesian. Essentially your filter will learn that news goes from good to neutral, and that javascript mouseovers go from bad to terrible.
However, this isn't what Joshua and the rest of MS are working on. His stuff is much more in the area of modifying SMTP so that untrusted clients have to perform some calculations before their email is accepted, or pay a few cents. My guess is it will fail since it doesn't account for zombie PCs but I'm sure he has something planned for them.
Well they weren't really a spam company, they sold software that allowed you to generate spam messages. I was going to do some telephone sales for them, cold call their market (I know, it's evil but I was calling corporations, not individuals, and I needed some cash) but after I got a copy of their software and became familiar with it's capabilities I felt icky, like I stepped in something, I couldn't in good conscience work for them. It had been presented to me as a customer contact software package - but it had too many little sneaky features that marked it to me as spam software, (built in SMTP server, throttle control on smtp activity so your ISP didn't get mad at you, and a bunch of message generation/tracking options) or at least there was nothing stopping customers from using it in that way, no matter how the company described their product.
The rock, the vulture, and the chain
They blocked the block function for microsoft messages in hotmail.