Slashdot Mirror
Researchers Claim "Effectively Perfect" Spam Blocking Discovery
A team of computer scientists from the International Computer Science Institute in Berkeley, CA are claiming to have found an "effectively perfect" method for blocking spam. The new system deciphers the templates a botnet is using to create spam and then teaches filters what to look for. "The system ... works by exploiting a trick that spammers use to defeat email filters. As spam is churned out, subtle changes are typically incorporated into the messages to confound spam filters. Each message is generated from a template that specifies the message content and how it should be varied. The team reasoned that analyzing such messages could reveal the template that created them. And since the spam template describes the entire range of the emails a bot will send, possessing it might provide a watertight method of blocking spam from that bot."
Is it coming?
Hooray for the good guys! Now if they could find something similar to fight viruses.
Free Martian Whores!
Sure, it will work "perfectly" for about 2 days, until the spammers change their methods to work around it. This is an arms race; there is no "final solution" (although modifying the email protocol to allow authentication of the sender's address would be a big help.)
I've abandoned my search for truth; now I'm just looking for some useful delusions.
Unplugging the ethernet cable DOESN'T COUNT.
"Effectively perfect" overstates this claim in a big way.
So what happens when botnets start adjusting the templates?
Knowledge = Power
P= W/t
t=Money
Money = Work/Knowledge so the less you know the more you make
"effectively" = "not quite good enough to actually work"
Go green: turn off your refrigerator.
If you have the botnet's source, then it's results are obviously predictable.
1. Mash up dubious quality meat. 2. Insert into can.
And since most devices will download updates and things automatically, new templates could be discovered and pushed out as well. I'm sure there will be some work around that the spammers will figure out, but hey, I'm up for most anything that will cut down/stop/prevent spam. I am also still a fan of the 'kill them until they die from it' club when it comes to spammers.
I will shred my adversaries. Pull their eyes out just enough to turn them towards their mewing, mutilated faces. Illyria
So it still needs to see a certain volume of spams in order to figure out the template. Then it reacts to the template. Then when the spammers figure out it's uncovered the template, they change the template. Spam will exist until the fundamental nature of e-mail operation changes.
You keep using that word. I do not think it means what you think it means.
Take it to the limit, everybody to the limit, come on, everybody fhqwhgads.
The researchers are seeking to infer the hidden distribution of spammers' find-and-replace tactics, rather than simply trashing emails with "pen1s" in the subject.
Correct me if I'm wrong, but haven't hidden markov models been around for decades?
"I'd just like to emphasise that taking a million years isn't a metaphor here..." -Rich Bradshaw
A team of hackers from Russia are claiming to have found an "effectively perfect" method for countering spam blocking technology. The new system deciphers the templates Spam Blocker is using to filter spam and then teaches spam generators what to write.
I don't believe any spam filter that advertises 100% accuracy, especially one claiming to do it by figuring out the spam email 'templates'
Sure, I'll bite:
This group advocates a:
(X) technical ( ) legislative ( ) market-based ( ) vigilante
approach to fighting spam. The idea will not work. Here is why it won't work. (One or more of the following may apply to the particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
(X) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(X) Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
(X) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(X) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
(X) Eternal arms race involved in all filtering approaches
(X) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
(X) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
(X) Ideas similar to this are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
(X) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
(X) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(X) Killing them that way is not slow and painful enough
Furthermore, this is what I think about them:
(X) Sorry dudes, but I don't think it would work.
( ) This is a stupid idea, and they're a stupid people for suggesting it.
( ) Nice try, assh0les! I'm going to find out where you live and burn your house down!
I am officially gone from
Err, what if I, as a corporation, blew out a spam that effectively incorporated a template unique to that which my largest competitor uses in their newsletters or customer communiques (or at least close enough to get my competitor blacklisted far and wide)?
(it would take a shedload of doing, but certainly not impossible, and if it could be done, would make for one hell of a cheap and easy DoS).
Heuristics is great and all, but go too deeply, and I can see it opening up a small but pretty scary can of worms.
Quo usque tandem abutere, Nimbus, patientia nostra?
Formatting! Please use some proper formatting! my eyes are bleeding from your wall of text!
Spoiled by html fail.
And BTW the spammers are just going to change the way their templates work. Make them more... evolutionary.
http://michaelsmith.id.au
Hah, leaky disciplines.
Fuck systemd. Fuck Redhat. Fuck Soylent, too. Wait, scratch the last one.
and then the researchers discovered the Halting problem and pretended it didn't exist.
...and that is all I have to say about that.
http://jessta.id.au
As long as there is money to be made in spam, spammers will continue to send spam. This "discovery" does nothing for that. Indeed it just dedicates more CPU time to trying to identify spam, which is just another way that internet users shoulder the cost of the profitability of spamming.
I've said it before, and I'll continue to say it - spam is an economic problem. Until something is done to address the money that spammers make, they will continue to find ways around these "effectively perfect" "discoveries".
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
I, too, have designed a flawless spam filter. It works under similar principles, will filter 100% of incoming spam, will generate 0 false positives, and it's super easy to use:
if(is_spam(message)) { delete_message(message); }
Their may be a grammatical error, misspeling, or evn a typo in this post.
Had there been no spam filters, we'd all receive about the same amount of e-mail spam as we receive in the postal mail world. Instead, the spam industry spends it's time trying to break through spam filters -- and they do so with volume. Upping the ante further just doesn't help. So now you'll encourage spam without templates. My grandmother's just never going to have a chance.
As a researcher in the academic side of the Information Security field, I can't help but notice a significant increase in the level of puffery and misleading promotion of research results. Self-promotion obviously isn't new, it's just that as the amount of newspaper-assisted promotion increases, the level of accuracy has dropped significantly. And more importantly, researchers seem much less apologetic about it. It's generating some real blowback.
The best recent example I can think of is Vanish, a cryptographic system for "destroying" data that was proposed out of University of Washington. It's not just that the system was broken a few days after it was presented, it's that this relatively minor result got more press than all of the perfectly legitimate crypto-systems research that was going on at the time. In fact, during the same time period a guy named Craig Gentry solved a major open crypto problem --- namely, how to compute on encrypted data --- and it got a fraction of the press coverage.
Not that I'm saying these researchers specifically asked to have their invention described as an "effectively perfect" solution to preventing spam --- which I guarantee you 100% it is not --- but that by going out on a University-encouraged PR junket, they've more or less encouraged this kind of coverage. This kind of stuff is damaging; people should describe their work as what it is. They've developed a technique that is highly effective at filtering /current-gen/ spam generators, in the lab. It won't stop all spam, and it's not effectively perfect, since spamfiltering is by nature an arms race. But of course that's not how it's going to be presented. In the long run this'll just make people more jaded with our field.
Creators recieve chance to increase wang size in 3...2...1...
The word "Perfect" neither requires nor allows a conditional modifier. "Effectively Perfect" makes no literal sense.
This makes it unsurprising that their approach seems uninspired. For example, who says the template cannot change? What if their template matches real email notes? What about image spam?
Email and Spam are like global thermonuclear war: the only winning move is not to play. (Spam will only go away when email does)
-d
"Here Lies Philip J. Fry, named for his uncle, to carry on his spirit"
There's this race between spammers and researchers. It seems from the article that spammers had been ahead of researchers for awhile by figuring out how to modify their emails in such a way that the spam filters wouldn't catch them. The article claims that research has caught up and figured a way to detect this. This spam filter greedily exploits attributes of today's spam, not tomorrow's spam. It seems a bit early to start saying, "Our program that's trained on today's Spam will catch tomorrow's spam!" Doesn't it seem intuitive that the spammers are going to find another way to get their email through this spam filter?
The best spam block ... still comes in a can.
And thanks to Top Gear and their arctic special for permanently engraving in my memory the image of a shotgun blasting a can of spam.
As a former manager and an "email direct-marketing" firm, I should point out that the spammers can increase the amount of complexity/variation in the templates by a wide variety of techniques, including rearranging paragraphs instead of just letters, making parts of the message optional, performing syntactic modifications of the included text,... Each new minor modification starts a research effort on the detecting side. The cost of detecting spam will rise much faster than the cost of generating spam.
If you try to outsmart the spammers with this, you will lose. Complexity favors the spammers.
Furthermore, bad will always win because good is dumb.
Note that the "good guys" revealed their methods immediately after discovery, which means the "bad guys" can start looking for a workaround. The "bad guys" won't make the same slip.
Honestly, I have to say between all the various filters I have or have written, I don't get a whole lot of spam. What I -want- though, is a way to identify it more reliably before my mail server even has to accept the message. With the current protocols, you can simply only block so much based on IP ranges or whatnot. There's a point where you have to accept the message to analyze. Sadly the only way we're likely to increase the chance of dropping the connection before receiving the message now is for the protocols themselves to change from the ground up. And as everyone here knows, that's highly unlikely to ever happen.
Ahh well...
I dunno. Are we sure they won't simply realize that if using a template to procedurally generate unique email for spam is ineffective after a short time, they can use a template to procedurally generate templates to generate unique messages.
Spammers send spam because it makes them money. It makes them money because people are stupid. The question is: why are people stupid, and how can we make them smarter? I would argue that spam is an educational problem.
From reading the article, it would appear that this method can successfully block 100% of spam emails that are identified as spam. How many emails will get through while the data is gathered to determine which emails are spam...is inconvenient to this statistic so you should just ignore that for now.
- To find right spamming botnet, study internals, find "templates" and voila - use it to prevent current spam messages! Or even simplier - to collect several millins of spam messages, analyze them all and find template. Than apply template and voila - problem is solved! Seems like thinking model of Homer Simpson.
It seems like "fails to account for (X) Asshats" is *always* the case.
Is it true, that perhaps "no one expects the asshats!"
I RTFA and they tested it by giving it 1000 spam e-mails by the same bot and after that it recognized the spam sent by that bot with 100% accuracy. This means NOTHING. I could bet a nice sum of money that if you give a traditional, learning spam filter 1000 e-mails sent by the same bot and flag those all as spam, it can then recognize the bot's further e-mails as spam. Real enviroment doesn't work like that, however. You have a large amount of very different spam bots and their templates which is what makes it so difficult. In addition, you have loads of regular mail, some of which might somewhat resemble the spam e-mails but still be completely legitimate. And in real enviroment, some people eventually flag legitimate e-mail as spam but some spam isn't flagged as such.
The fact that their test was so limited implies that this was simply a test. A proof of concept for this kind of approach, one could say. I doubt they actually intended to this be a solution that ends spam.
As a co-author of this work, I should be clear that we never suggested that we have a perfect spam filter per se, simply a new tool that has the benefit of being orthogonal to existing techniques. For _existing_ botnets, our filters are extremely good, but the paper is also quite clear about the variety of ways that spammers might try to evade the approach.
..I don't see it.
Exactly. They just make the subtle changes in templates less subtle. They have a reason (money) to get around the blocking, like they already do. This isn't going to be some effectively perfect solution.
Case closed.
Divining the template seems to depend on analyzing numerous messages. Presumably, only very large mail servers (or an aggregated network of smaller servers) would be able to collect enough messages to rapidly divine the various templates. It sounds like a small or medium site could not benefit from operating the analysis software themselves; they would not have sufficient spam volume (from each template) to rapidly divine the template.
because good is dumb.
nah, just lazy. think of the other stuff good is doing. like beer and women. and uhm yeah. stuff like that.
I've said it before, and I'll continue to say it - spam is an economic problem. Until something is done to address the money that spammers make, they will continue to find ways around these "effectively perfect" "discoveries".
There is always a demand to get a message out to n% of x hundred thousand people for cheap. You can't realistically stop that. What you can realistically do is increase the cost of getting those messages out. Treating spam as simply an economic problem won't work.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
I think it would be much more effective as well as cheaper to give free counseling to any one who ashamed of the size of their penis.
No thanks, I'm good.
"The ferrets, they're every where I tell you!"
Typically there were many solutions to SPAM that includes:
1. Filtering by keywords and phrases
2. Bayesian (or more complicated AI filtering)
3. Trusted Domains and its flipside: blacklist database
4. Domain keys
All of these are defeated by various means:
1. There are many ways one can say "Viagra", "ViAgRa", "V1@gra" or just "that blue pill women talk about". So word and phrase obfuscation is used to fight the spam.
2. Bayesian filtering is defeated by seeding the database with valid text (For example, how many of you saw The Lord Of the Rings passage on the bottom of your email? This effectively defeats the probability counters, but given enough email training, as long as the size of the given group is magnitudes higher then the size of the sample tested, poisoning technique would be less likely to work. There are some AI developments in filtering technologies to enable to overcome this problem.
3. Blacklists are easily defeated by "botnets" and all other distributed spamming "nets".
4. Domain keys are implemented by "Yahoo!" and "Gmail" and others, creates a net of "trusted" domains and smtp relays, but still defeated when a spammer infiltrates a given domain (as evidenced by recent influx of spam from gmail) and uses its trusted status to send out spam.
So, I am surprised noone has ever talked about scrapping SMTP protocol completely and replace it with something a lot more secure. AMTP (http://amtp.bw.org/) is a good start. My personal opinion is that there is supposed to be several important features present in the new protocols.
Since 95% of email is spam, just block it all.
No one will notice the statistically-insignificant 5% false positives.
Effectively perfect, no. If nothing else, for certain classes of spam(especially phishing) the money or perception of money can be good enough to keep actual humans at the keyboard.
/dev/urandom, is that you both need to peddle whatever it is you are peddling and look vaguely like a human constructed message. If the researchers can, in fact, target messages that bear signs of being generated from a given template, the spammers will be forced to be looser in generating messages from templates(which increases the risk of garbling beyond comprehension, or being flagged by filters looking for highly non-human output) or step up their game in terms of natural language synthesis.
However, the reason you use templates, rather than word salad or the first 100kb of
Furthermore, bad will always win because good is dumb.
Nice saying, but if you need an excuse for being bad, you're not doing it right.
The force that blew the Big Bang continues to accelerate.
Probably the worst slashdot article I've ever read.
Improve at backgammon rapidly through addictive quickfire position quizzes: www.bgtrain.com
Not in the same level of detail; but, when your business model is spamming, you inevitably end up sending thousands of samples to loads of ill-vetted email addresses, some fraction of which are either being operated as spamtraps, or are in the possession of users annoyed enough to forward samples on.
Your algorithms can, and often do, remain secret(unless one of your black-hat buddies cracks one of your cracked machines); but you'd be a lousy spammer indeed if the results of your technique weren't widely available.
Asshatitude always applies because you can never anticipate the next step in asshatitude evolution. They will always find new and innovative ways to be asshats.
Bill Gates proposed (though I'm not sure where the idea originated) an escrow service for e-mail. You get to set the amount you're willing to pay me to read your e-mail. I can, at my option, take that money.
If I know you then you do a token $0.01 amount and I don't take it. If you're spam, then I take it and you lose money.
The only real problem I would anticipate is that spammers are in the same camp as those with fraudulent credit cards and the like. They would probably just fund their spam with fraudulent sources. Since it would be an escrow system, people would still receive that money, but it wouldn't actually be costing the spammers anything.
One side benefit of such an escrow service would be to finally open the way for micropayments on the web.
Can't lay bricks made from shit, you know.
Presumably, only very large mail servers (or an aggregated network of smaller servers) would be able to collect enough messages to rapidly divine the various templates.
If they don't graylist, and if they insist on putting the spam filtering in between accepting and placing in the mbox/maildir.
If they wait for enough other small sites to aggregate the info, and then spamfilter mbox/maildir instead of spamfiltering the inputs to mbox/maildir...
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Spam filtering isn't very hard, if you see the email for a large number of accounts, as Gmail does. The one characteristic that spam must have is that it's sent in bulk. The commonality across receiving email accounts gives it away. The only hard part is recognizing the commonality, which is already working rather well. This is just a new technique for recognizing commonality.
Recognizing spam for a single account is tougher, because you don't get to see the "bulk" property.
Police wont put up with it??
Despite what many may think, police are human too,
and they get spammed too,
and they are sick to death of it too
What you can realistically do is increase the cost of getting those messages out.
The proposed "Spam Blocking Discovery" doesn't do jack shit to accomplish that goal. The people who install the spam filters aren't going to buy anything that was spamvertised, anyways. Meanwhile the spammers will continue to adjust their methods to get around the filters that are installed at the ISP level so that they can get their messages out to more people who would be interested.
This craptacular "Discovery" is just another round of whack-a-mole. Hopefully at some point people will finally get tired of this (and realize that they are getting nowhere by doing it) and actually work on the root economic problem, instead of just addressing the symptoms.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
I'd say it's 'effectively perfect' against the templates it's targeting, not against all of them. Since templates are the best way to get around a bayesian filter, you 'could' limit spammers to manual spam again, which is a big crap-shoot. Until they develop a new method (which isn't the target the filter is 'perfect' against).
Write your representatives! Repeal the 2nd Law of Thermodynamics!
This is actually quite simple once you've got the basics in place. It reminds me of a program I once wrote that could crawl a website and it would find out the templates used, identify the actual content, title and other blocks. Some postprocessing was required though, but since most e-mails are a lot simpler than webpages, I suppose this can be done completely automatic for spam. And probably indeed "effectively perfect". As long as spam is template-based, that is.
0x or or snor perron?!
Yeah, this idea is great. . . until it starts blocking out legitimate emails which really are confirming orders shipped by Amazon or other retailers, newsletters that people really were wanting to get, and other info that 'looks' like spam, but isn't.
This is why, while I use spam filters, I would never rely on them to delete email. All I want filters to do is punt suspect spam off to the Junk folder, where I can review it later, or find the email I was expecting which got mis-classified.
Spammers send spam because it makes them money. It makes them money because people are stupid.
What does intelligence have to do with wanting to enlarge my manhood.
...is the manual filtering by the recipient. Actually, scratch that, I've deleted emails that were clearly legitimate. Ah well, as long as it adds to the arsenal.
I agree, it's an arms race. Technology gets better, they get creative. The 'root economic problem' is no better. You can't stop people wanting to make money. So long as you can send messages to anybody for absurdly cheap, people will pay to get their message out. Law enforcement can't do shit about that. Spammers get paid regardless of the success of the product. Email needs to flow freely.
Either email has to fundamentally change on a technical level to defeat spam or 6 billion people need their brain re-wired.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
"Most" of the time it will be full proof.
Reminds me of the Sex panther advertisement: 60% of the time it works all the time.
Spammers send spam because it makes them money.
Agreed.
It makes them money because people are stupid
Not directly. The spammers themselves are paid by moderately smart people who are selling products online that are often of questionable legitimacy. While some of those customers are stupid, there are generally fairly crafty individuals making money off of the customers along the way.
The question is: why are people stupid, and how can we make them smarter?
You could ask the same question in the light of why 419 scams work, why old-school pyramid schemes work, etc. Money can make smart people pretty dumb at times.
I would argue that spam is an educational problem
You will not succeed in educating the problem away. Unless you want to impose some sort of requirements for users to access the internet (a la driver's licensure), you won't succeed at educating all the users and getting it to stick. It's like trying to design a better mouse trap; nature will just make a better mouse and then you're back to square one.
If you want to make a meaningful difference in the spam volume, you need to stop the money from flowing to the spammer.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
my eyes are bleeding from your wall of text!
Did you register that user name just for your post? I got a chuckle from it. BTW, the eye surgeon who performed my vitrectomy is Dr. Odin (no shit). The first hint that I needed to see him was my eye bleeding -- internally.
You can imagine the humor I see in that comment, and I thank you for it.
Free Martian Whores!
Competition spurs innovation! Prepare for the next generation of spam, now procedurally generated!
Similes are like metaphors
I have 100% protection from SPAM system.... do not get an email account... PICK UP THE DAM PHONE AND PHONE THE PERSON OR BETTER YET GET OFF YOUR ASS AND GO TALK TO THEM IN PERSON.
The in person part is the best because if they start spewing spam you can just punch them in the face - especially when they start talking to you about your dysfunction and how to improve your man-hood.
You wouldn't even think of running a contest to defeat this blocking method.
Stephan
http://stephan.sugarmotor.org
I still say Gmail is the perfect spam blocker, mabye 1 spam gets to my inbox a year.
I don't care if YOU JUST LIKE SCREAMING or if you're lysdexic or you english not good o si hablas otra idioma completamente, no quiero ver sus mensajes nada.
Is that really THAT HARD to implement? Really? If it's not at least 95% proper, coherent english, I just don't want to see it, spam or not. Plus there's a very short list of people I have any contact with on other continents. Aside from them, if it's not from north america, I don't want to see that either. Nothing from nigeria. Block it all.
I have the perfect spam filter:
Block everything!
100% of spam gets blocked.
And just like my filter, the filter this company has created will cause a lot of false positives. At work, we send a lot of internal mail that's all in about the same format because it's easiest to read that way. It's a lot more formalized than spam is, so it would definitely be caught first.
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
Now suppose my account were compromised and you got this exact message from my personal email, where the jpeg is a Viagra ad. There is absolutely nothing there for your spam blocker to latch on to, unless it parses the content of the jpeg itself. Anyway, blocking stuff like this would lead to unacceptably many false positives.
I have a feeling that Google does something like this, which is why it's so convenient for so many of us to have our email sifted through Gmail's filters. (I'm not saying it's wise, only that it's convenient.) If there's one thing that Google have, it's lots of data.
Well, as I see it, the "Asshats" here is not a reference to the spammers. I see the spammers as an anonymous mass that will always exist. Faceless and brainless, acting only on instincts.
The asshats would be more like people that do not want to send spam, but who see it as an interesting challenge to disrupt the supposed spam-protection, or to abuse the actual protection system just to use up resources.
c++;
(X) technical ( ) legislative ( ) market-based ( ) vigilante
Has anyone ever suggested all of these? The government offers a contract and clears the legislative barriers to a company making vigilante robots which would hunt down and kill the families of all spammers while making the spammers watch?
Assuming these robots can fly, have powerful metal claws, and cannot be stopped, I can't see any problems on your checklist.
( ) Spammers can easily use it to harvest email addresses
() Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
() Users of email will not put up with it
( ) Microsoft will not put up with it
( ) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Nope. None there.
( ) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
There are currently laws expressly forbidding the construction and operation of mass murder machines, but that's why I suggested we get rid of those laws.
( ) Ideas similar to this are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
(X) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
( ) Killing them that way is not slow and painful enough
I do realize some wouldn't trust the company controlling the deathbots, which is why -I- would be the governing authority once they were operational. You can trust me because I promise to only kill you if you're related to a spammer.
If the researchers can, in fact, target messages that bear signs of being generated from a given template, the spammers will be forced to be looser in generating messages from templates(which increases the risk of garbling beyond comprehension, or being flagged by filters looking for highly non-human output) or step up their game in terms of natural language synthesis.
Much of the spam I've seen already qualifies as garbled beyond comprehension. At best there may be an obfuscated URL that I won't copy/paste to my browser. Spammers have already gone down the road of messages that make no sense whatsoever. I suppose it is because they just need to be able to say they made the run to whoever hired them and perhaps attest that X number weren't rejected out of hand.
was put into catching and killing this scum as was spent on filtering, these assholes would have such a limited lifespan no one would risk it.
Rick B.
how about the spammers using fragments from Gutenberg books ? Or fragments from blog posts ? ... What is spam, after all ? I am trying hard to send David Horowitz the the spam bin, but then the guy manages to get out of it after a while ... I have tried unsubscribing, tried "spam"-ing him, even tried to beg him to let my mailbox live peacefully ... for me it's spam, for him it is enlightening the dumb masses and the work of his life ...
Good is Dumb
Take their algorithm and design a botnet that doesn't trigger it. I presume they're claiming this is essentially, hehehe, impossible.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
And like most arms races, the opposition will swarm over your latest creation to reverse-engineer it, redesign it, build countermeasures, and neutralize it.
And count on the spammers being subscribged to your service. They'll get your filters as soon as their victims do. The iteration delay will be infinitesmal.
Refer to the previous posts as to why it still won't work.
deleting the extra space after periods so i can stay relevant, yeah.
And how do you accomplish that? I agree that education is a hopeless endeavor, but getting at the money doesn't seem to be any easier.
Spam isn't kosher anyway!
This filter is not intended for end users, but for people who administer mail servers. It wouldn't work for end users anyway, since it needs to chew through thousands of emails in order to learn the patterns.
Obviously it's over-hyped, but it could be useful.
But if the US government was to threaten the US based credit card companies that process every single one of these transactions there would be no more money, and no more spam.
Obama may not be able to win the war in Afganistan, but he could stop spam tonite by threatening Visa and Mastercard, and it would not even need a single water board or nuke. Though I personally would vote for bankers (and spammers) to be waterboarded. Preferably on prime time TV.
Sent from my ASR33 using ASCII
The only reason bad tends to win is because out of the entire range of things to do, there are more bad things than good. Someone bad thus has many more options open to them, whereas a good agent needs to avoid things that do harm, by definition, and thus faces more constraint. It's the same reason the universe moves toward greater entropy, but with actual disorder replaced by social/cultural/moral/ethical disorder.
OK, I've been waiting for the opportunity to suggest my idea. It's completely back-ward compatible with existing technology, and it will only help filter spam:
The biggest problem I have with SPAM is unverifiable email addresses. Try replying to a SPAM message... most of the time it's a bad address. So, fix that, and you have solved 99% of the SPAM problem. (BTW, I don't consider email from verified email addresses to be SPAM. Why? Because they can be held accountable if they shouldn't be sending to you in the first place.)
The other big problem I have is that when I send email to friends and I have a link inside, it gets marked as SPAM. So, having a way to trust my messages would be good.
So here's how my idea works: Every mail server also hosts a "verify server." Every email client can connect to a domain's "verify server." (Compliance is completely voluntary for either the client or domains. And if you don't comply, you just don't get the benefit of trust.)
1. You write an email message with your email client (Outlook) and hit 'Send'.
2. Your email client generates a 50-digit, randomized alpha-numeric psuedo-unique serial number.
3. The email client includes this serial number in the message as a header or some other tag.
4. The email client then connects to it's own "verify server", logs in and gives the server this serial number and its email address.
5. The receiving email client then receives the message. It sees the serial number in the tag, and contacts that domain's "verify server."
6. The receiving client asks the "verify server": "Was there a message with serial number '5sd56123515baCesieoo25il2oigloowldogi255i289602d0d0g' and from 'bobb_sledd@gmail.com' ?
7. If the "verify server" says yes, then we verified that the message at least originated from that email address.
What do you think?
"They said I probly shouldn't fly with just one eye," "I am Bender. Please insert girder."
I agree, it's an arms race. Technology gets better, they get creative. The 'root economic problem' is no better. You can't stop people wanting to make money
I agree with you that far.
Either email has to fundamentally change on a technical level to defeat spam or 6 billion people need their brain re-wired.
However I see a third option on this. Another option would be to come between the spammer and their financing source - essentially attacking their margins. Spammers themselves are generally part of multi-layered machines, and each layer has its own cost and profit margins. One spot in particular that I have previously suggested cracking down on is the domain registrars that make spamvertising work.
In particular I am thinking of the registrars that sell the domains that are spamvertised (as opposed to the domains that produce the spam), as well as the domains that provide DNS for the spamvertised domains (which are generally owned by spammers, spamvertised profiteers, or others in cohorts with them). The spammers (and their customers) rely on those registrations because it allows them to move quickly from one ISP to another without loss of web service - even if the ISPs are on different continents.
Unfortunately, our friends at ICANN have chosen to make that work much easier - easier for the spammers, that is. When individuals will be able to buy their own TLDs, then the (very flimsy and largely toothless) regulations that we currently have on TLDs go right out the window. Spammers will be buying and selling lifetime domain registrations with obfuscated (or completely absent) registration data, making it impossible to determine where the money came from or is going.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
I did not know that Steve Gutenberg wrote books. I thought he was just a skilled actor. Go figure.
Dude, that's brutal. There was a Metalocalypse where that happened to the audience at a Dethklock concert.
Man, you bled from your eyes. That's hard core.
You are welcome on my lawn.
The truth is that spam has been successfully fought by filters without compromising legitimate email. Furthermore as Paul Graham had stated, spammers have been forced to yield in smaller text-based messages or in-line images.
In particular,
(X) Mailing lists and other legitimate email uses would be affected
Possibly but the probability of losing legitimate email by modern heuristics is (proven) smaller than the probability of accidentally deleting it when it is mixed with spam.
(X) Users of email will not put up with it
They do, sometimes without their knowledge
(X) Many email users cannot afford to lose business or alienate potential employers
They would lose more without filtering. See 1st argument.
(X) Asshats
How ?
(X) Eternal arms race involved in all filtering approaches
(X) Extreme profitability of spam
And also extreme profitability in having a working e-mail address.
(X) Bandwidth costs that are unaffected by client filtering
This isn't the mid 90s anymore.
(X) Ideas similar to this are easy to come up with, yet none have ever been shown practical
The practicality of heuristic filtering (SpamAssassin etc) is proved by its transparency. Even old e-mail clients such as Outlook 97 can filter out email marked by X-Spam headers. Gmail and the rest of the privacy traders do it for you automatically.
(X) Why should we have to trust you and your servers?
Run it locally. Mozilla Messaging does.
(X) Feel-good measures do nothing to solve the problem
Age old forms copied from the newsgroups can't be used as arguments anymore. Time to be creative again!
(X) Killing them that way is not slow and painful enough
But cutting down their profit is.
A team of computer scientists from the International Computer Science Institute in Berkeley, CA are claiming to have found an 'effectively perfect' method for blocking spam. Reply by forwarding this message to ten people within one hour and see what luck befalls you! Failure to do this may bring tremendous ill fortune to you and your immediate family.
Say hello to my little sig.
That's an interesting thought. But how would they distinguish a domain from a spam domain, and what would be the registrar's incentive to police that?
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
I don't know the entire history of the checklist (which can be found here). However, my understanding is that it solved a longstanding problem on various spam-fighting mailing lists where someone would post what sounded like a bright idea for fixing the spam problem once and for all, which invariably had several of the listed drawbacks to their proposal. So the checklist was created to provide a quick way to explain exactly why the proposed plans didn't work.
I am officially gone from
Can't lay bricks made from shit, you know.
Quick, someone call the Mythbusters! If they can polish it (and they did) then I'm sure they could make...
wait for it...
A SHIT BRICK HOUSE!!!
Your post advocates a
() abusive
(x) checklist
() clever
(x) tired
approach to mockery. It won't work because
(x) the joke is too old
(x) nobody has the patience to read the whole thing
() we are above that
...the fraction of a fraction who really think(!) that whatever is being peddled will actually work, and make a transaction ... thus making spam profitable and continuing the supreme obnoxiousness that it is.
"Good news, everyone!"
You want us to wait till March?
As long as there is money to be made in spam, spammers will continue to send spam.
But if the US government was to threaten the US based credit card companies that process every single one of these transactions there would be no more money, and no more spam.
Which transactions should they block?
It's also important to keep in mind that spammers don't make money from selling V1AGRA. Spammers make money from other people who want to make money by selling V1AGRA. The distinction is important because it doesn't really matter whether money can be made by selling shady products or not. As long as there's a sucker who *believes* they can make money by selling the shady products, the spammer has a customer. When that one wises up, there are 10 more waiting.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Furthermore, bad will always win because good is dumb.
I see your Schwartz is as big as mine
So instead of delivering e-mail right away, you hold it for a bit, 5-10 minutes or something. Your system then looks at the mails it gets and uses template matching to find spams and axe them.
I did not know that Steve Gutenberg wrote books. I thought he was just a skilled actor.
He's only a start because of the stonecutters.
http://michaelsmith.id.au
Filter mail for proper English grammar and syntax. It has the benefit of identifying spam and those annoying "thought for the day" emails you get from your mother.
I know that from a server admin viewpoint the percentage of email that is SPAM is huge - in the 90%s or so. But from an end user perspective (mine), this has already been solved well enough. At home, I use both Gmail and Outlook 2007 (for my ISP provided account). I almost never see SPAM in either of those accounts. At work, we have Exchange server and some sort of filter in front of it (I don't know what they use for sure, but there is something). I never get SPAM there either. Would it be nice to keep these SPAM notes from bouncing around the internet tying up bandwidth? Absolutely. Now that I don't get SPAM in my various inboxes do I give a crap anymore? Nope.
There's already a near-perfect method for spam filter: Hire someone to read your email. If it's spam, they delete it. If it isn't, they pass it on to you to read.
UTF-8: There and Back Again
Start a spam campaign of your own. Advertise free samples of m3dz. Only stupid people who buy from spam will be stupid enough to buy from your spam. Send arsenic pills.
Customer base for spammers will quickly approach 0. The gene pool will get a needed dash of chlorine.
UTF-8: There and Back Again
What we need are "Moles";
Computers that interact with spam botnets as if they were zombies, but really arent.
EG, they either run from behind a VERY closely monitored network, or they run a very specially crafted version of the botnet's remote control software:
The purpose is to use the botnet's own rollouts of "enhancements" to be detected, and implemented immediately in the filter (with some sanity checks to avoid this being exploited A-la apple and it's "Leaks" policies.)
This way if the spam botnet rolls out a new template every X hours, the filter will automatically update right along with it, thanks to the Mole.
Since the botnets have to have an automated system for deployment of templates, and many botnets have already been compromised for the "how" of this being done, it makes sense to capture the update process, then compute every possible message that the template can produce, then MD5 hash them, and then check incoming emails to see if they match any of the hashes.
This would stop pretty much all but handcrafted spams, and make the utility of using botnets much lower.
It would also be a good proof of concept for how one can use the knowledge of how a botnet works to make a zombie work for you, without going on the offensive and assuming control over computers you dont own, and the legal quagmire that this entails.
Granted, "major" updates of the botnet to better obfuscate its update process and communications protocols would cause momentary lapses in coverage while the Mole gets analyzed, but this *IS* an arms race, afterall.
how would they distinguish a domain from a spam domain
It would actually be pretty straightforward. Say you receive spam from "big-als-viagra-shop.com". You look up that domain and find the DNS servers that depend on it. You report the spam to the registrar of those DNS servers, and send the same report to (the replacement for) ICANN (because they are spineless profiteering cowards). Currently most spamvertised domains get DNS from a very short list of DNS servers, sold by a very short list of registrars. When more than a certain number of complaints are received for the DNS domain, the registrar is required to shut down the domain or lose their accreditation.
However, ICANN can only regulate accreditation for registrars involved in certain TLDs. As we've seen before, ICANN recently approved the future sale of TLDs themselves, which throws all regulation out the window and opens the flood gates for new spamming opportunities (made possible by new total freedom from registrar regulations).
what would be the registrar's incentive to police that?
Well, for the next several months or more, ICANN has the ability to strip registrars of their accreditation. Once TLDs are sold then it all goes out the window and we'll see a new game unfold.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
I originally posted it here in 2002. Note how dated it is (e.g. no smartass comment about CAPTCHA).
Some mathematician (I forget who) had his graduate students send back cards with forms like these to people who sent in attempted proofs of Fermat's Last Theorem.
How do you entice registrars to spend the energy to act on this?
How do you get every registrar to do that?
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
This is precisely a workaround the fact that the black hats* do not disclose their algorithm. This is basically a reverse-engineering algorithm. It'll certainly be beaten by next gen spammers, but it's (possibly) a step in the arm race.
Also, I think the white hats weren't dumb to "reveal their methods". That's what open source is all about.
* beats "bad guys", no? ;)
I remember when I first heard of Bayesian filters (here on Slashdot, as I recall) the article was very optimistic about how the filter would be nearly unstoppable... Something to the effect of "to beat this adaptive filtering system, spam would have to stop looking like spam, to the point that it would also cease to be effective" - as if any spam message had some intrinsic "spammy character" inherent in its word chains, and any attempt to change this would also prevent spammers from formulating an effective advertisement... This obviously failed to account for all the ways spammers have found to undermine Bayes filters over the past several years, but I was very impressed with the idea at the time.
Bow-ties are cool.
I asked my undergrad complexity theory class to prove that testing equivalence of regular expressions is PSPACE-complete on their final exam last term, well one or two figured out the proof strategy and almost pulled it off. In other words, even if you consider only very basic models of computation, you still cannot easily determine whether two programs have the same function, i.e. you cannot recognize if your template really captures their template.
I imagine their system works quite effectively upon the current botnet's, which likely don't even support regular languages. I doubt however they'll effectively deal with a botnet that incorporated say the Dada Engine : http://dev.null.org/dadaengine/ Are the spammers that clever? Almost surely yes.
p.s. An interesting legal strategy might be simply advertising spam services and prosecuting the companies that buy it, sure that's entrapment and they'll walk, but you might gain enough evidence for civil cases by ISPs, and you'll ruin their reputation in any case.
until the spammers find out.
so the spammers will write an algorithm that alters/permutates the template to fool the sensors. clever fail is fail.
I can't find the paper anywhere. This is just hype! Templates are relatively similar, however, the trick is in figuring out what the polymorphic engine that injects the content does. Currently, there's quite a bit of research being done uses machine learning techniques. I guess this paper is supposed to do the same thing, learn from some training data and filter other spam. So, there's a learning phase and a testing phase. Current research shows that for the short term these techniques are quite successful having really low false positives 0.002 (AutoRE at Microsoft). However, when looking at spam say 6 months down the road unless you keep learning from labeled emails spam/not-spam, you won't be able to make good decisions because of the volatile characteristics of spam. Also, you've got to be concerned with the Real-Time implications of these learning methodologies and noise. Most campaigns last around 5 days, however, will a user have to wait for 5 days to receive an email? Probably not, so this technique may yield low positives in a longer period than the short term.
hunt down and kill the families of all spammers
Good thing you didn't include friends, 'cause then there'd be a reason to add another check-box to the list:
( ) Kevin Bacon
Of course, it does depend on how you define families. Especially if you're a Creationist.
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
Is what's done by my rather small ISP with the help of Red Condor software. Spam is far easier to handle at ISP level in their case. Just look for incomings that target nearly all the customers (which they get plenty of, as they host for all those insecure windows boxes). At first they wanted to charge for this, but I suggested they do it in self defense and they thought it was a good idea (along with selling virus cleaning services for the malefactors and bots). They do a really nice job, very low error rate, and sort the odd things into two categories -- things that have known viruses in them or links thereto, and things that *might* be ok at the top, and they send this to you as email -- just a list that you can view or delete. You can then whitelist things that might be a mailing list you actually want, but it's rare that they catch that, because not that many people subscribe to the same ones. This mainly detects viruses and things targeted to say, more than half their tens of thousands of users. Works.
Eg, what makes spam a problem is sheer numbers. This algorithm detects sheer numbers, and it really works. In 3 years it's had exactly one false positive, 3 misses that got through, all recent. That's darn good, and now I just delete the summary without looking closely.
Now, the 195 spams a day I get are mostly because they answered to "HELO" for a long time after everyone knew that was stupid (we are in the sticks here, and I wouldn't take the sysadmin job for them). So it's their fault, but in another way it's genius -- now anything that targets their entire user base, or nearly is automatically spam and easy to detect as can be. Bam, no spam.
This method is reactive rather than predictive. When a new spam blast is sent out, it can only start filtering after it reverse engineers the template, a critical delay which means a higher % of spam messages will get past than against virtually all predictive methods. Meanwhile the spammers are just going to generate the template itself from a template.
Universities are increasingly employing PR idiots to put things out.
When they decide to do something about your field it's time to cringe and hide in a cupboard once it gets regurgitated into the newspapers. I'll bet the MIT materials science people are still trying to live down the report about the bulletproof skintight superhero suit they were supposed to be working on.
Or put it more succinctly as was done in The Usual Suspects:
"They realized that to be in power you didn't need guns, or money or even numbers. You just needed the will to do what the other guy wouldn't."
You can never go home again... but I guess you can shop there.
>how about the spammers using fragments from Gutenberg books ? Or fragments from blog posts ?
.. uh .. I mean, I can consider .. from a purely technical point of view, that is. ;-)
Gee Thanks ! That's great ! Have you got any other related ideas I can try
Brain power! Just like in that contest for the TSA, which offered up prize money for the best idea for securing air ports, I said forget all the high tech stuff, and spend the money on paying just above decent wages and "security" (rent-a-cops) will actually want to keep from getting fired. Nothing beats an "above decent" human brain (when compared to technology).
So, get a bunch of people filtering spam for us. Who cares if you think it's an invasion of privacy? They don't know who you are and you'll never know who they are. And besides, people already know more about you than you even know! For example, my company is basically a contractor company, so we go places that most people in our line of work just shouldn't be. I have a co-worker who regularly tells me about law suits and patients medical histories they find interesting!
So as long as their only purpose would be to "read for spam" and not "look for illegal activity" it would work!
But then again what do I know?
My abilities are only limited by my imagination
I did this years ago. By planting bogus email addresses from my mail domain on the web and feeding these addresses directly to a statistical spam filter I would get instantly updated on the changes in spam templates. Because the spammers were feeding the filter themselves I get a very low FP-ratio and extremely tight spam blocking.
I'm curious to read the paper, as I've developed a template detection mechanism and presented it on the CEAS 2008 conference -- but I used it just for characterization purposes, not filtering. 3 distinct spam campaign templates in the middle of this figure: http://spammining.speed.dcc.ufmg.br/spammining/images/tree2.jpg
this is already being done ... just look into your google spam folder ...
Based on the fact that not a single response (at least that I have found) seems to be remotely taking this idea seriously, do they then earn the stupidest article / idea award from slash in recent months (years)?
Perhaps they should at least get the lest original idea.
Living in Chile
The point of this is not to "win" the war on spam, but to force the spammers to convolute their message with sufficient ingenuity that the messages become unintelligible to the morons who purchase the products or buy the penny stocks or launder stolen funds. The point is to convolute the spam until the only option left to the spammer is to render an ice cube in ASCII art containing a subliminal giant wiener.
It's not an arms race against the spammers, who are plenty smart, it's an arms race against their customers, who for the most part are as dumb as a bag of glass hammers.
You advocate a:
( ) technical ( ) legislative ( ) market-based (x) vigilante
approach to fighting spam. The idea will not work. Here is why it won't work. (One or more of the following may apply to the particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)
( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
( ) Users of email will not put up with it
( ) Microsoft will not put up with it
(x) The police will not put up with it
( ) Requires too much cooperation from spammers
( ) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
( ) Anyone could anonymously destroy anyone else's career or business
Specifically, your plan fails to account for
(x) Laws expressly prohibiting it
( ) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
( ) Asshats
( ) Jurisdictional problems
( ) Unpopularity of weird new taxes
( ) Public reluctance to accept weird new forms of money
( ) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
( ) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
(X) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
( ) Extreme stupidity on the part of people who do business with spammers
( ) Dishonesty on the part of spammers themselves
( ) Bandwidth costs that are unaffected by client filtering
( ) Outlook
and the following philosophical objections may also apply:
( ) Ideas similar to this are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
( ) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
( ) Sending email should be free
( ) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(x) Killing them that way is not slow and painful enough
Furthermore, this is what I think about them:
(X) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and they're a stupid people for suggesting it.
( ) Nice try, assh0les! I'm going to find out where you live and burn your house down!
Snowden and Manning are heroes.
Oh he's not saying it can be done, just that it isn't an educational problem. Making people smarter would not solve the problem, even if you could. Stopping money from flowing to the spammers would, not that it's possible.
financially go after spammers. The are doing it to make profit right? by click throughs. So sue them for more money than God has to financialy bankrupt them and then put a legal restriction where they can not own or operate or even live in a house with someone who owns a computer type device that connects to the Internet. The kind of thing they have now for felons and guns. Sure it doesn't work all the time, but it stops a lot of shit.
NOBODY EXPECTS THE SPAMISH INQUISITION!!!!!
On another note, this sentence is to bypass the caps filter.
-1 disagree is not a modifier for a reason. -1 troll, flaimbait, redundant, overrated are NOT acceptable substitutes.
If they have developed a perfect spam blocker I'll kiss their fanny on main street at high noon. I hate spam!
I've got a better one. "Don't talk to botnets"
No rDNS? Goodbye? rDNS looks dynamic? Click.
If your mail server only talks to properly allocated static IPs, most spam simply vanishes.
To paraphrase George Lucas via David Prowse "Don't be too proud of this technological turnbuckle you've constructed. The ability to destroy a botnet is insignificant next to the power of pain."
The Russians have won. They have made the world a cesspool of distrust, greed, fear and hate.
Ha! Look, everyone, it's a bot! You can tell, because no actual human being would have the words "Steve Gutenberg" and "skilled actor" in the same paragraph. Quick, add him to the magical template detection AI's list!
We use Thunderbird with the Enigmail (OpenPGP) plugin at my office to cryptographicly sign and/or encrypt our email.
Our SPAM filter consists of simply rejecting all unsigned e-mail messages.
One exception is that external e-mail addresses can be whitelisted (with approval) to allow for email from companies with no email authentication in place.
It's a bit of a pain at first, but everyone at work agrees that it's a small price to pay when you consider the alternative (inboxes full of spam).
I've never recieved a spam e-mail message at work.
New employees create PGP keys during orientation (or else they can't send any e-mail).
One by one I'm convincing my friends and family to cryptographicly sign their messages (tech-savy ones love the Idea).
Soon I hope to get zero spam at home too.
[sigh]... If only the rest of the Internet authenticated their email we could all have Zero spam in our inboxes.
Its actually quite funny that it takes a team of "computer scientists" to attempt an approach that any slashdot reader knows instinctivly to be foolish and doomed to failure.
Until their smarter than humans all anti-spam efforts are ever sure to accomplish is to make the Internets mail system increasingly unreliable for legitimate business to the point of being absolutely useless.
Internet mail needs a complete overhaul. Listen up Berkley computer science heads... Conceptually the only scheme that has a chance of ever being practical is requirement to obtain "permission to send" ..
HELO, can I send you an email?
(User is notified and accepts)
THANKYOU, here is my email
Permission is likely to be in the form of a signing request that can be shared with others ad-infinitum by linking the trust chain. Once permission is granted its always a bi-directional grant by default and the keying material is used as a basis for mandatory message signing and optional message encryption.
The receiver has the capability of revoking their signature if its abused by an organization or its sub-assignments. Once revoked permission to send will need to be re-obtained for that signature and any sub-assignments. This disentangles the email address and prevents you from being a spam target even if your email is posted publically.
Users are in full control and as with typical PKI you can set recursion limits and EKUs to specify if/how your permission to send can be given to others.
If your smart about it you can overhaul the SMTP protocol and maintain IMAP/POP3 client compatibility. IMAP extensions can be used to manage permission to send/signing mumbo jumbo and a compatibility mode can provide interactive email prompting from the new server.
Yes you can still be spammed by millions of zombies asking for permission to send you something so there is a careful balance of what information should be conveyed in a request and valid modes such as prearranged passwords or specifically requested information before permission to send requests will even be acceptable.
I would much rather have that and have some assurance WRT who I'm talking to /w built-in ability to go secure/encrypted when needed. Its not foolproof but at least it does not require a trusted third party and if your smart about it there is some chance it won't even have to be a disruptive transition.
Block templates and we'll just end up getting more Markovian spam, or something generated on some other yet-unknown technique.
OMG, take a look at this adorable picture of Jake playing with Mike's puppy!
[attached jpeg]
Mary
Now suppose my account were compromised and you got this exact message from my personal email, where the jpeg is a Viagra ad.
"Hmmm...oh, hey! Looks like I got an email from Mary!"
*double clicks email*
"Ohhh, cute pup... wait. Well, it's not a puppy, but it does remind me that I have been suffering from erectile dysfunction lately. Thanks Mary!"
"A government is a body of people usually -- notably -- ungoverned." -Shepherd Book
...then compute every possible message that the template can produce, then MD5 hash them....
I was with you up until this point. Just consider a line of, I don't know, say 64 random characters. A complete MD5 DB just of this alone would make existing MD5 rainbow tables look tiny.
The idea of "moles" it not bad though (nor new). It would be possible for the SMTP server to look at a message and ask itself (or, more likely, an external filter) "Could this message have been constructed from this template?". As notes elsewhere though, it would be just another arms race.
Connection closed by foreign host.
Isn't this exactly the same thing we do when creating spamassassin regexps to block spam ? I know I do this several times a week, trying to identify common elements on spam to create effective filters that work despite the variations.
What part of this is news again ? We've all been doing this for years.
morcego
It's very easy to write a parser for text, and weight words by the common-ness of usage. Just download any of the widely available freebie e-books written by a Star Trek fan writer, and weight all the words therein by the number of times used.
With that simple hash table, you can combine the numbers together and then generate random numbers within the total range, and kick out the associated words! Statistically indecipherable from real text (because that was the source!) and you kill heuristic anti-spam filters to the curb for good.
It's a simple algorithm - I could write a simple script in PHP that would do this and crank out hundreds of thousands of unblockable messages per hour on a 5 year old Celeron. And the war will *never* end. Even if you can filter improbable word associations, or any other factor or set of factors, those factors can then be figured into the random word generator.
It's an endless game of cat and mouse and it will never end, so long as we're using heuristics to try to figure out what spam is and isn't.
But if we *are* using heuristics to try to read spam, that's probably the single biggest funding driver behind AI research that there is! In essence, we have an intelligent-design genetic algorithm at work trying to come up with the best algorithm for defeating our own intelligence!
What better way is there to fund the development of our eventual intellectual replacements?
I have no problem with your religion until you decide it's reason to deprive others of the truth.
It seems to me that all we need to do, is charge a penny for every email sent and it is held in "escrow" at the ISP.
If the receiver getting the email, reads the message (and you can automate what a user action constitutes "accepted" mail). Then the penny is returned.
So for most people, you'd have maybe $5 for all your emails that you might send in a year. And likely, if you're only NOT going to get back a receipt for maybe 50 cents in a year.
For a Spammer, sending out a thousand emails in the hopes that they get one sucker, costs them $10 upfront money.
So, is it worth it just to pony up a tiny bit of money and end this problem? We don't need tricks, hacks or technology. All emails go through an official router, and only people with credit in the account get through. Of course, you can still have "anonymous" email -- just so long as someone pays for it, either through anonymous donations or ads. But, so that we don't go insane with avoiding our own email, because it's a chore that wastes so much time -- I would definitely spend a penny an email.
All the illegitimate groups that want to SPAM you, can go around the routers -- it's a free market. But your email will be set to ONLY receive from the verified routers with the escrow account. For every SPAM i receive, I will get a penny. It should be more for the pain and suffering, but at least I'll know the SOBs had to waste some money.
>>"ad space available -- low rates!!!"
This is another word that doesn't mean the same thing to everyone, and therein lies the root of the problem.
I have web site clients who have almost begged me to send out "newsletters" to "prospective client lists", and I
refuse outright every time, because if one of my sites gets flagged as spammy, they all do, and we're all down
the shitter together.
"But it's not spam, it's targeted advertising", they tell me, "and we put an opt-out link in anyway, so it shouldn't
cause any problems".
BZZZT - wrong, please advance to the next webmaster, this one's not playing ball.
The thing is, there ARE some people who would genuinely like to receive updates on real estate offers recently
arriving on the market, and there ARE some people who want to buy Viagra and Cialis, and there ARE some
people still ignorant to believe that a jar of cold cream with an "exotic" smell will magically make their mini-willy
into jumbo-tube-steak-king.
So long as someone clicks on these emails and buys, then the advertisers will still pay the spammers to send
out the "advertising material". Frankly, I like to receive the discount brochure in the mail detailing European
holiday specials, and I don't subscribe to it - it's spam, and I like it. But I would prefer not to waste anyone's
time by continuing to receive discount women's underwear adverts - thanks, but it's really NOT my thing.
We will NEVER stop spam, but we can develop a filtering mechanism so those of us that don't want X flavour
spam will only get Y flavour instead. Build a better white/grey/black list, and it has a chance of working. But it
really MUST include the white list section so that there is a genuine incentive for the advertisers to instruct
spammers to stick to the rules. Otherwise we are all left with ALL of the spam.
(X) Joe jobs and/or identity theft
You're just giving botnet operators an effective means of setting up a hitman agency, and they don't even have to get see the victims.
So they learned how to install SpamAssasin?
An SQL query goes to a bar, walks up to a table and asks, "Mind if I join you?"
> I don't know the entire history of the checklist
I'm pretty sure it comes from usenet originally and is older than email spam. I believe it may also be older than the web.
Cut that out, or I will ship you to Norilsk in a box.
LOL, nice!
Single Women and Men Looking to have sexual encounters! - sexually explicit - PRESS HERE to meet single women or men for sexual encounters: (MUST BE 18 and OLDER) http://zimbrlen.com/tr.php?85012+ooga@siteware.com This is going to sound crazy, but there is a dating site loaded with thousands of single women and men right in your city that are looking to meet and date new people this week. These single women and men are not interested in playing games. They know what they want, and they will be very upfront with you. You have nothing to lose, and the time of your life to gain! PRESS HERE to meet single women or men for sexual encounters: (MUST BE 18 and OLDER) http://zimbrlen.com/tr.php?85012+ooga@siteware.com
Inventor, Artist http://www.Rubber-Power.com
> individual rights aren't related to human genetics, but rather to the organism's sapience
So do you look at individual ants, or the entire anthill to determine the level of sapience ?
What a depressingly stupid machine.
Until, that is, they change the template's format and then we start back at square 1.
We've been doing this with Message Sniffer since day 1 (many years now!). It's very effective. In fact some of the first rules we coded (abstracts we call them) are still active and effective. Along the way we've developed automation to help us see key pieces of these patterns in real-time, and bots to take advantage of other vectors, but the process is still fundamentally the same for template driven spam.
It should be noted that this vector breaks down badly when the spam template is strongly modeled after legitimate messages - such is the case for many phishing spam.
Clearly it is not a complete solution either -- I'd say that it's good for more than 40% of new spam campaigns and less than 70% on average.
All you have to do is identify a large quantity of emails that you're already 100% certain are spam generated by one particular system using one particular template. Then this process can analyze it and deduce the template so that in future you can... be 100% certain that certain emails are spam generated by one particular system using one particular template.
Yawn.
> Gmail and the rest of the privacy traders do it for you automatically.
Gmail has a great success rate for me... It stops about 1000 spams a month and maybe lets through 2 a year. I've had maybe 2 false positives in all that time (that I've known about.) One was a note from my ex-wife, so maybe GMail knew what it was doing after all.
Whoops, it was 2003. It seemed like longer ago.
It is quite hard to imagine a AI that would demand the right of not being an slave. Really. At least the first batch will be created for a propose, why do you think they'd want to not fullfill their propose? (Who would create them on such a way?) If anything, such AI could demand the right to work more, if we ever try to limit it.
Rethinking email
> And also extreme profitability in having a working e-mail address.
It is not profitable to spammers that you have a working email address.
But, it actually does work. I've got nearly a thousand messages in my junk mail folder and very few false positives. All of the false positives are mailing lists which can be easily whitelisted when I feel like it. And there's been no spam getting through. That's a result that demonstrates the effect it has.
Additionally Google crowd sources the problem, when one user places the spam into the spam folder, any and all identical messages also get placed into the spam folder in other accounts. Meaning that the spam only gets seen one time, even though Gmail may have many thousands of copies in various accounts.