Slashdot Mirror
New Kind of Spam 'Un-Training' Filters?
Zaphod2016 writes to tell us the Wall Street Journal is reporting that email in-boxes are under a new kind of spam attack. This new spam has confused many people due to its lack of advertising, viruses, or request for personal information. One popular theory is that these innocuous blocks of text, often drawn from popular literature, are being used to "un-train" spam filters to allow more malicious spam through in the future.
I got some with advertising images. Let me guess, you strip images from suspected spam.
As a hobby, I play around with ways to classify spam. Not much of a hobby, but I find the problem interesting.
Lately, I've also been trying to use my vectorspace engine to classify spam.. so these sorts of things might get in, but only because they fall into the general category of readable text...
I've also been thinking about building a GPL tool to provide "sound-based" classification sort of like a "one second orchestra" playing in harmony/disharmony based on the content.
Regardless of the engine I use, I still have to dig through my trash bin every few days to make sure nothing good slipped through.
meh
I've been getting 3 or 4 of these a day for at least a month now. The text can
always be found in some file of an old book provided by the Gutenberg
Project, which is making non-copyright texts available through volunteer
effort.
I think the theory about using this stuff to untrain spam filters is very plausible.
But it's difficult to see how it will work. There's no common text among these
e-mails; in order to send effective spam, there'll have to be at least some text which
is the same across multiple mails, and that will tend to expose it.
At work our spamassassin bayes filter has heavily trained on English text always being spam.
This is because English is not our local language, so almost no business communication is in English and most of the spam is.
This indeed sometimes causes false positives when English language mail has other spam-like properties as well, and the added 3.5 points from the Bayes filter pushes it above the limit.
This again shows that you should not use solely a Bayes filter as spam blocker.
The text block spam is very common WITH images . I suspect that what happened is some lame spammer got a BIG botnet contract, sent out his spam, and forgot to include the image.
Test your net with Netalyzr
Here are some excerpts of this type of spam from my school's mail filtering system, Mail Marshall:
..."
... so I'm not sure how this will "train" the filters.
"One cannot bring children into a world like this. She tried to get hold of things by the right end anyhow. She stood her upright, dusted herfrock, kissed her. Perfect nonsense it was;about death; about Miss Isabel Pole. And of course she enjoyed life immensely. He has his penny, he reasoned it out
Here's my favorite, with some bizarre non sequiters:
"Yes, we are dirty, said Maggie, looking at her; she was in her day clothes. Prejudiced;narrow; unfair, he repeated, tapping her hand with his finger. The light from the engine lit up a quiet group of cows; and a hedge of hawthorn."
Thing is, the spam detection already catches it
I have seen some of these slip though for a while I think the only purpose for them is to get some neophyte who is confused by them to send back a "WTF?" response thereby confirming a "live one". I suspect after that the floodgates open. I am sure that we will see many more attempts to circumvent filters. After all, weasels abound.
Assuming that GMail has 10e6+ users, and assuming that it doesn't matter if one mail with spam slips through into 100 mailboxes, where they will be marked as spam immediately, does GMail still have to learn about spam? Or are they already able to md5sum spam messages and filter them out?
On the other hand such collaborative spam filters might have existed and not been sufficient before...
My home spam filter does not seem to be affected much. I run dspam which has a feature in that over time it will forget words if they are not used in spam. Since the text is usually different or random, it does not have any significant effect on generating false positives. In the years I have been running dspam with tens of thousands of emails, I have only gotten 3-4 false positives.
By having a baysian filter forget over time, it also helps shrink down the database and helps it adapt as the contents of spam change over time.
Of course I also use other spam blocking techniques, like using realtime black lists (RBLs) and blocking a number of Chinese subnets... I should add tpnet.pl and Verizon as well.
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
Just like the cryptic number sequence radio/voip 'stations', this could be a method of communication.
We see so much Spam everyday, everyone takes it for granted, and everyone runs 'filters'. If I wanted to secretly inform agents to begin operations, a select quote from a book sent as spam to hundreds of thousands of people would be perfect. Everyone ends up on spam-lists, and recieving spam is a passive process, so its even more anonymous than public web forums.
man is machine
Spammers till have to tell you these two crucial pieces of information. If they're selling Viagra, they have to make that known to you somehow. If they're selling anything (and not just trying to increase brand awareness, which is a separate problem), they have to tell you how to contact them and buy whatever crap they're peddling. They can make this very hard to discern via obfuscation, leet speak, image substitution, etc. But the contact information ultimately has to boil down to something meaningful and unambiguous, or there won't be any sales.
So the solution is to recognize and ignore spam based on either or both of these criteria. Ultimately, a collection of trusted humans need to review a message and say "this is spam, alright", allowing the filters to recognize the contact information (phone number, email address, web site, etc.) as spam.
I'm not too worried about spam that tells me to "Drink Coke!", I don't get much of that.
it's a blue bright blue Saturday hey hey
*shudder*
I can't even imagine what sort of lasting damage one could do to one's, uh, member.
Eureka! That's how to stop spam. Educate people with a campaign reminiscent of the Speed Kills campaign, so that people understand they could permanently damage their penis by taking unregulated pharmaceuticals from shady online stores hosted on 0wn3d pcs.
If you mod me down, I shall become more powerful than you could possibly imagine.
It is such animportant element, you see, that duration
of time. I consider twelve hours a substantial measure. So I ran along
the drive and upthe steps and into the house, but did not see either
Mrs. Iobserved:Your Excellency is not easily satisfied. And I marvelled,
and said:How comes it that I have hitherto been deaf to these
distressfultones? Il passe sur la route, mais toujours en sens inverse.
For a mental state such astheirs, appetency rather than instability is
the right word. Which reminds me that the old adage about let us eat and
drink, forto-morrow, etc. Mais odonc est la vie, sinon dans le peuple?
They lamented dismally among themselves in many tongues:How I suffer!
Take that little one on Lzards, for instance;or, in the other volume,
the bizarre Joies Noires.
NPR covered this issue this morning and had a guy from project Gutenberg read a few sentences like this. I have a degree in literature (I know, shocking), and I thought to myself that this would qualify as good dada.
I'd rather have someone respond than be modded up.
I see the war of SPAM as an escallation war. Each side escallates its response to the other sides latest counter move. At some point, the system is gonna break, and we haven't quite reached that point.
......
The real problem with SPAM is what I call "hidden costs" associated with it: the extra bandwidth, the cost of increasing filtering technology, the labor costs, oppotunity costs due to filtered legit emails
Only real pain is going to stop SPAM. Pain on the SPAMMERS or on those paying for the priviledge of being spammed. When the system gets to its breaking point, someone is again going to suggest a payment scheme for email, one that effectively denies the spammers emails, but allows free (both kinds) (or very low cost) email to the masses.
I think I have just a solution. Email Broker Tokens.
When Email accounts are created, the creator is granted a number of tokens along with the service he is paying for. These Tokens are then attached as part of the email being sent, and are collected by the recipient, who adds these tokens to his account. For each email sent, a token is exchanged with the recipient who then has tokens to use for further emails.
Since the "net" usage for MOST people is, on average, even, this system will work for "most" situations. Those people who legitimately need to send more can buy them from their provider, or buy them on the open market, from people collecting excess tokens.
This exchange system will effectively reduce all spam to semi legitimate commecial email. The quick rich V14gr4 / C14li5 and nigerian / lottery scams will be left high and dry. They will then have to pay to send email.
I realize that this is going to require a entire overhaul of the email system as we have it today, but that also provides a side benifit eliminating zombied SMTP servers on Windoze boxen. At some point, the system is going to break under the load. Might as well start planning for it.
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
Outlook 2003 also blocks these types of messages but I find GMail misses all of them. I have encountered 3 types. One is where all of the text looks random, but if you turn on HTML viewing the positiong of the text is different and you can see the hottest stock. The second is blocks of common lit, like the Hobbit, but the spam message is contained in 3 images whose file names are random words. The third is the text without the images, I assume the spammer screwed up in this case and forgot to attach the images. I also find that all of these types of spam message for me have been about buying stock.
I dont think this is the case, as Ive been getting these sorts of emails for at least 3 years (looking back at the spam archive I keep to train from) - random blocks of legible text, blocks of psuedo english (words are correct but theres no effort at sentence structure), even jokes on their own. I got intrigued by this about 6 months ago and wrote a few scripts to see if it was just a broken spam client forgetting to add the payload, but your average 'with payload' spam doesnt seem to match these emails, theres practically no similiar 'with payload' spams in my archive with these blocks of text.
I always wrote it off as baysian filter poisoning.
As long as I can remember I always received spam that didnt have an advertisement, didnt have contact information at all etc.
Some spammers spoof their emails so well you couldnt contact them if you were interested in their crap. Many times it is a bit of text with a click here (but nowhere to actually click ) etc.
I think the spammers are just idiots. It is amazing most of them actually managed to get the software working and send an email because of how craptastic their messages are (not disguised, just junk)
The phrase "more better" is acceptable English. suck it grammar Nazis
I Recommend that you subscribe to a couple of english language Mailing Lists (or Yahoo Groups), which you can then filter and move to a mail subfolder of their own easily through the Subject line or From Address. That way you can have good english non-spam mails going through your Bayes daily.
As a Slashdot discussion grows longer, the probability of an analogy involving cars approaches one.
My limited experience is that whatever filtering Hotmail uses has been allowing lots of Spam to slip through in the last few weeks.
Anyone else?
How's Yahoo & G-Mail been doing?
I use gmail, and although it's let one or two pieces of spam through in the last week, it's always been near 100%.
I get 50-100 email a day on gmail.
I've seen this for maybe 3 years. Right after Bayesian filtering came out for Spamassassin.
Maybe longer.
I'm seeing spam that uses relatively coherent passages from literature of some sort as a way to deliver an image that is usually a pitch for some stock, lottery, or bank scam.
Rick
deleting the extra space after periods so i can stay relevant, yeah.
There are far better methods such as neural nets or support vector machines. You can for instance see a comparison of classifiers on a simple visual 2-d problem to see how inferior Bayesian filters are to other more sophisticated algorithms.
I'm a SpamCop user, and I have noticed they've been letting through a bit more recently.
Though, that's a bit offset as of late, due to the fact that I've been getting a lot MORE spam recently as well. I usually find a good 40-50 messages sitting in my held mail after about 8-12 hours.
It's getting better slowly as I report more and more of the stuff that makes it through though.
I wrote up a small clip on the C/R system I use which is built into CPanel (BoxTrapper). If you know when to manually add things to the whitelist, there's nothing more effective. http://journals.fotki.com/airjrdn/Tech-Ramblings/e ntry/sqsggqkqrtq/
My Tech Posts on Twitter
CAN-SPAM killed spam as advertising, in a way that neither the Direct Marketing Association or the anti-spam groups expected. CAN-SPAM has criminal penalties for forged headers, but doesn't restrict "legitimate e-mail marketing", which is what the DMA wanted. But with valid headers, spam filters can immediately discard spam. The result is that "legitimate e-mail marketing" attempts go directly to the bit bucket today. Notice how rarely you see a spam from any legitimate company any more. (This assumes you have reasonable filtering.)
With the legitimate businesses gone, spam became a branch of crime. To be a spammer today, you have to commit felonies. Which means a risk of doing jail time. The famous "Buffalo Spammer" went to jail in 2004, and gets out in 2011. Jeremy Jaynes was sentenced to nine years in prison; he's out on bail pending an appeal, but sooner or later he's going to do those nine years. There's a Registry of Known Spam Operators, and law enforcement reads that list. Most of the people on that list have had visits from law enforcement.
Spammers have tried moving offshore, but that's not working as well as it used to. Few countries want to be known as spam havens. Even in China, it's getting harder; spammers have had to move from the developed coast to more remote provinces, where Beijing has less presence. ("The mountains are high and the emperor is far away") Operating offshore draws the attention of the investigators who follow money-laundering, terrorism, and drug-dealing. There are people doing this, but the risks are high.
What's left is what you'd expect - wannabe crooks, as in any bad neighborhood. They're not very good at crime. They're not making much money. They're what cops call "regular customers". They're a problem, but not a major threat. Those are the ones sending out useless spam.
Actually, on second thought, #3 shouldn't be "last date seen." It should be "how many messages have been received since the last time this term was seen."
Are they only using text form old books that are out of copyright or otherwise have authors who have left the planet? Spam filters that can differentiate between modern and older writing styles should be able to handle this, especially if they can tap into databases of classic liturature. Spam filter would search on the text and if it matched classic literature, then it is spam. This could be a real problem for people who use legitimate email to discuss classic literature.
If you must moderate, please moderate as irrelevent, not something bad, because I'm sure someone will find this interest
I've been getting several 419-type spam emails over the last few weeks, up from zero in the past. I thought back, and the only thing that has changed is that my email address appeared in plaintext on the front page of slashdot. Not saying that it's the problem, but it is interesting in the timing.
Dammit Otto, you have lupus.
I'm not exactly sure, but I think the problem with these spam getting further and further away from being legible is caused by market forces. I think the spammers get paid for delivering spam, NOT how many responses/click thrus/sales they get. So, if they blast out an e-mail to you and don't get a bounce, that counts as a successful delivery. Thus, they don't really care what's in the body of the e-mail. They did their job, and they get paid for the delivery.
That's all I can figure, because if your average person is so stupid that they respond to spam, then I think they aren't probably smart enough to figure out what "Viggra" is.
I think I've confused Yahoo by applying for a mortgage. So I've been getting lots of legitimate mortgage and real estate-related emails, and it's been starting to let through a few related spams as well.
Other than that, I haven't been getting any more stray spam than usual. Maybe once a week I'll get one (that's not mortgage-related) that the filter misses.
Then there are the ones that go to email lists that I have filtered to other boxes besides Inbox... Since you can't pick when the spam filter works, it always works AFTER all your others, and so I get all of these. *sigh*
Warning: Apple/Nintendo fangirl. Likes her electronics cute & cuddly. May be rabid.
I have started seeing a rise in SPAM getting through GMail's filter featuring Arabic-looking characters. I wonder if this type of spam white-noise is designed to overwhelm the governemt's email spying programs.
NOTE: I attempted to cut and paste an example of the characters. But when I went to preview my message, they were cut-out.
NPR had almost exactly the same story yesterday, including an interview with Paul Graham.
What makes swarms of reporters cover the same thing at the same time, when the situation has been going on for months or years? Are they all on the same IRC chat channel?
My biggest problem of late is that one of my business clients got their email account pwned, and now the spam is slowly starting to drip into my formerly-pristine business account. At least, I'm pretty sure that's what caused it.
Information wants to be free.
Entertainment wants to be paid.
You just want to be cheap.
My company uses a spam filter in Microsoft Exchange. It filters about half of the mail I get from mailing lists I have signed up for (mostly Apple development mailing lists) to the spam folder. About half of my actual spam is sent to the spam folder and about half gets into my inbox. Sometimes mail from other people I work with gets marked as spam. Basically this filter would do the same thing if it just threw about 1/3 or 1/2 of all the email I receive into the spam folder randomly.
.mac email address and use Mail in Tiger on MacOS X. The junk email filter does not have very many false positives, but it still lets a lot of spam into my inbox.
I also have an Apple
On one of my machines I am doing a trial with Spam Sieve. It is doing a better job, but has had misses and false positives, but it is better than either Apple's filter or the useless Exchange filter.
Avoid Missing Ball for High Score
Of that f[o]rbidden [t]ree whose mortal taste
Broug[h]t d[e]ath into the Wo[r]ld, and all our woe,
With loss of Eden, till one [g]reater Man
Restore [u]s, and r[e]gain the bli[s]sful [s]eat,
S[i]ng, Heavenly Mu[s]e, that, on the [s]ecret [t]op
Of Or[e]b, or of Sinai, didst inspire
That shepherd who first tau[g]ht the chosen seed
In the beginning how the he[a]vens a[n]d earth
R[o]se out of Chaos: or, if Sion hill
Deli[g]ht thee mo[r]e, and Silo[a]'s brook that flowed
Fast by the oracle of God, I thence
Invoke thy aid to my adventurous song,
That with no middle flight intends to soar
Above th' Aonian mount, while it [p]ursues
T[h]ings unattempted [y]et in prose or rhyme.
Just an idea...
I've about become convinced that the Viagra and other drug spam must be funded by the drug companies themselves. Not because they want us to buy the drugs from the spammers, but just because the constant barrage of email adds up to advertising impressions.
Obviously the emails I get for this crap are so badly done, nobody would actually expect me to buy from them. If I was actually trying to make money selling bogus drugs through spam, wouldn't I work harder to make it look legit? The phishing guys don't seem to have too much trouble making good looking e-mail - so why are the bogus drug emails so childish?
Because they don't exist. It's just advertising impressions. They've managed to get the word Viagra and Cialis in front of me a few more times a day, really cheaply.
Rather than send random garbage that, as others have said, bears no resemblance to the users' typical email, why not extract text from the domain's website? A large portion of spam goes to work addresses. Emails sent and received with these addresses often times contain the name of the company, major individuals, current products, industry jargon, etc. So google the second half of the address and insert blocks of text from the company website/related pages. It seems to me that such a method would be much more obvious and effective than using Project Gutenberg. Especially in the short term, the one which matters most in this case.
"Strangers have the best candy" -Me
"A fool and his money are soon parted."
What's the difference between some guy selling a tonic via SPAM and a tonic at the state fair? At the end of the day, not much, just that the spammer reaches more people.
My client-side email app does filtering on the header only. It also applies a few tests to the sender name and email. (Reads each header off the server, checks it out, rates it spam, not spam, or unsure.)
I get phenomenal accuracy without looking at the body, and it's quicker too.
Hal Spacejock: Science Fiction with Nuts