Slashdot Mirror
Spam Gets Personal
Vitaly Friedman writes "Two researchers demonstrate how much more effective spam could become if its authors used basic data-mining to personalize their messages. From the article: "North America, though no longer the world leader in spam production, still has serious potted meat problems. A recent research paper out of the University of Calgary suggests that those problems could soon be a lot worse if spam creators adopt a few simple data-mining procedures.""
If this isnt personalized, what more can I expect? :)
http://it.slashdot.org/article.pl?sid=06/04/28/181 1210
And not very accurate the first time, either. Since Mom probably isn't going to be sending me v1agr4 ads, it will be easy to find and clean the infected machines.
Thanks! just what I want spammers to know
University of Calgary!!!!!!!!!
The paper was funded by the Canadian NSERC (the equivalent of the NSF in the US). Aren't you Canadians proud that your tax dollars are going to research in how to make spam more noxious?
Are they also hosting some pages on their site to help me make anthrax or a nuclear bomb? How about how to pick up under age girls.
Seriously; do the spammers NEED any more help?
Humor from a Genetically Molested Mind
Two researchers demonstrate how much more effective the AIDS virus could become if only a few basic modifications could be made to personalize the attack on the immune system.
Th US most definately is the world leader in the production of spam
treat the disease not the symptoms
And while we are at it, lets publish a paper telling people how to do a better job money laundering, or new way to smuggle cocain into the country.
You don't know serious potted meat problems until you've seen my kitchen sink.
Paul Grosfield - the quicker picker upper.
I'm lumping this article describing how spammers could be yet MORE annoying with the Fox News special reports in which Geraldo Rivera details how many people could be killed if "terrorists were to jump this 6 foot chain-link fence and put a couple buckets of toxins in this bay-area resvoir".
Thanks - hope those spammers/terrorists have TiVo and a notepad.
Scott Richter, are you getting all this?
to become blue ribbon morons: help the spam industry by doing their research.
Personally, I like the light SPAM. It's not nearly as salty as the regular.
...if they advertised things the average person wants.
I'm ready to give up on email because of the spam load. At this point I'm seeing mail servers with significant load simply for spamchecking, graylisting, and hanging up on bogus inbound connections. Face it, smtp doesn't work. It's a tragedy of the commons happening right in front of all of us.
We need something different that focuses on point to point authentication of hosts and users. Frankly, hardware DRM or immutable hostids build-on to motherboards might offer at least a host authentication solution. Not a popular suggestion, I know...
The reason they don't do this now is that the spammers doing it are not geeks. They're taking pre-built scripts, modifying some parameters, and letting them go. They will keep doing this until those scripts no longer work, and then they will move onto newer ones. The only was this will happen is if some hacker gets bored, reads this article, and desides there's a lot of cash to be made selling just such a thing to the spammers.
Be real -- no matter how personalized an email gets, I'm still going to know it's not from somebody I know, because I don't make email my primary mode of correspondence and where I do, I can easily figure out that my mother isn't going to be sending me ads for Viagra.
Now, if they could make a Turing-capable spam generator, I'd be impressed.
GetOuttaMySpace - The Anti-Social Network
How else would they know my p3n1z i5 5m@LL?
"It's a wonderful idea. But it doesn't work." -- Tad Danielewski
fantastic. you've now told spammers how to defeat basically every statistical spam filter. now i get to attempt to teach the generally tech-clueless people in my life about pgp or equivalent so that i can automatically block all non-signed email. except i can't, because there are no online vendors / banking services / etc. that sign their outbound email, to the best of my knowledge.
just because you know how to do something like essentially unbreakable steganography in video sequences doesn't mean that it's something you need to share with the rest of the world.
So, that's why I get all those VIAGRA messages?
One thing to note, however... Once you start mining information from a Zombie (which -- to be honest has already been done), it makes it easier to identify the zombie and shut it down. (I.e. if I get a spam with information from mikie's machine, I'll immediately phone him and tell him to shut down and clean up his machine. Now mikeie's machine is unavailable to the spammers.)
I think that that is the real reason why zombie systems don't use data mining.... It's like an 'undercover' cop who fingers every low-level pusher-addict he runs into.... He'll never live long enough to get the information he wants on what goes on inside the biker gang's 'clubhouse'.
This is one of the things that I do... I wrote a filter that peels apart an email, removes the 'legitimate' IPs in the Received: headers collected en route, and attempts to send an email to the IP responsible for the source of the email. It usually takes them a while, but they will shut down the responsible zombie.
I stopped doing that for a couple of months, and my spam climbed to unbearable levels. I started using the script again a couple of days ago, and the spam I've been getting has already dropped noticably.
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
Every day I get quite upset by opening my reallife mailbox.
It's totally unacceptable: Buried below a ton of trash I find two seriously dangerous invoices with 4digit numbers in the red. If I ever miss out one of them I'd probably go to jail, but hey, why not throw another pizza flyer on top of all that, the planet sure can handle this and what else are those trees for?
Personally if I was going to choose I'd vote for e-mail spam just to get rid of this total waste of ressources.
There should be a LAW against this, and against buying from spammers, reallife or virtual.
Corn is no place for a mighty warrior.
Spammers could start targeting their messages just at people who fit a certain demographic.
And they could start sending their messages via USPS bulk rate.
This would surely bring about the end of civilization as we know it.
That's "Mr. Soulless Automaton" to you, Bub.
Fortunately for those who detest spam, the authors also present four new defenses that could help stop this newer, more personalized spam. First, e-mail archives can be encrypted, making it difficult for malware to mine them for information.
WOW - so I've got to accept that my computer IS broken into and encrypt even local data? Thank you very much - my computer would rather not be broken into.
Second, these archives can also be "salted" with false information such as spam trap addresses. Third, the authors suggest that all URLs followed from an e-mail client be viewed in a "sandboxed" browser that would prevent automatic downloads.
Sandboxed browser? Ok - they're joking. Who uses external content displaying in their mail? And anyone hasn't got a "HTML=+80% spam" rule in mail client yet, generated AUTOMATICALLY FROM EXAMPLES?
Finally, anti-spam filters can be adjusted to better screen for these types of attacks.
Care to elaborate?
Ok - this is all going in the wrong direction. Why shouldn't I trust *my system*? Why should I allow my incomming mail to use outside objects? I thought that people, who can build a natural-language-messages data mining / composing system can understand basics of home computer security...
Besides - if spam will mimic a friend's style and probably send mail as that friend - then you know exactly who to filter out and who needs billing for a "PC security" lessons
That should have been:
and attempts to send an email to the ISP responsible
(fyi: It involves a reverse DNS lookup and abuse.net records)
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
Or get one here http://makeashorterlink.com/?A25124D0D
It's deja vu all over again!
I can't speak for UKian snail mail, but here in the US critically important mail -- usually legal mail -- is sent return receipt requested. Meaning that someone has to sign for the mail, and if no one is available to sign one must go to the post office to sign and pick up the letter.
There is nothing analogous to that in email. Primarily because there is no mechanism to first ensure authenticity and then ensure delivery. A public-key cryptographic system that used hardware level keys (or key generation) could at least ensure authenticity point to point during envelope exchange. Knowing for certain exactly which host sent a message would mean being able to track down hosts sending spam. It would also mean being able to reject mail from specific hosts, rather than ever shifting IP addresses.
Perhaps you'd get less spam if you didn't display your email address prominently on a website in the exact format spiders are used to harvesting. Seriously, I get one unwanted email on a bad day, none on most days. I doubt yahoo has incredible spam filtering, so I'm not sure exactly why I get so little, but little things like obscuring the address can make a significant difference.
"Potted meat can go very wrong, very quickly, and we'll all suffer the consequences!"
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
These guys are from Calgary. A gram of prevention is worth a kilogram of cure. None of these barbaric, obsolete units of measurement for them.
Ah poop, I'm going to get modded offtopic again, aren't I?
I think that, moving forward, one of the core drivers of true artificial intelligence is goinig to be SPAM!
As algorithms become better and better at sending SPAM, combatting methods will become increasingly sophisticated.
Witness the Bayesian filtering phenomenon. Back in the day, who would've thought that a "learning" system would be needed just to determine what's junk mail?
SPAM is a side-effect of intense economic and evolutionary pressure - the value of getting your attention and maybe your pocketbook. Its pressure is relentless, and success is highly profitable.
I give it another 5 years before bona fide neural networks are commonly used to combat SPAM.
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Spammers have been personalizing content since day one... After all, if you don't have a flaccid penis, you probably know someone who does at any given moment. Who doesn't know someone who needs more money? Who doesn't want cheap drugs?
The whole point of the spam business model is that it's low-cost. Any filtering would raise costs compared to simply flooding the world with the same payload.
If spammers were in the slightest interested in addressing their markets, I wouldn't be seeing several thousand Asian-language spam per day addressed to a North American mail server. None of us would be seeing spam with hash-busters, mangled "Subject:" lines, and other filter avoidance hacks.
This seems like one more attempt to promote the idea of "good spam" for mainsleazers like Kohl's department stores.
Lacking <sarcasm> tags,
Dear Son,
How are you doing? I'm doing well. Have you refilled you Viagra recently? The dogs are doing great, but they don't have any Viagra. Btw son did you know buying your Viagra from kljhasdf.com could save you 90% of the pharmacy price? Well I just wanted to make sure you were ok.
Love,
Mom
This is my own personal opinion, but I think e-mail has to go in the direction of EASY TO USE crypto based authentication. This technology already exists (pgp) and is used heavilly by the computer security industry. it would make a lot of sense IMHO if EVERY e-mail from my bank was cryptographically signed using the bank's private key. Websites are encrypted and authenticated using public/private key cryptography (SSL) why can't the same thing be done for e-mail?
If Microsoft, Apple, Ebay/Paypal, Verisign, a few banks etc... got together, agreed to a SINGLE existing standard, and implemented it in a transparent and easy to use way, it might go a long way to reducing spam. Citibank could say, "all e-mail we send is cryptographically signed by Citibank. If you get an e-mail that is not signed by Citibank, then it isn't from us." Obviously there are still USARS out there who wouldn't get it, but i think this would be a big step in the right direction.
(P.S. Yes I know a variety of e-mail programs implement various crypto stuff already, but as far as I can tell, almost no one uses it or knows how to use it.)
there will always be a relatively small percentage of people who show maladaptive behavior. Just as there is a much larger percentage of people willing to take advantage of those unable to control themselves. It's criminals and their victims vs. everyone else.
The solution is not to be found in expecting *everyone* to change their behavior, because such an expectation is bound to fail. The solution is to be found in tightening up the mechanism behind data authentication and transport, both with technology and laws. Just like as was one with snail mail in the past. At one point the government realized that mail needed to be stamped, tracked from post office to post office, and then hand delivered by someone responsible. Well, we needn't charge to stamp email - but we certainly need to stamp it with an immutable ID, track its movements from host to host with immutable ID stamps, and then authenticate delivery at a specific host.
This can only be done with cryptographic hardware installed on every machine, and a new SMTP protocol. Sucks, doesn't it. Bye bye anonymity, but at least it would get rid of spam. Pick your poison.
Damn spammers hiring researchers to figure out better ways to get spam delivered. Don't they teach ethics anymore?
This also qualifies as a DUH! Of course if you send spam that looks like it comes from someone you know it has a better chance of getting through.
And yet, if you look at any posts about how Microsoft or Sony or whatever are trying to keep their software's flaws obscure so they don't get exploited, the Slashdot community generally rails on them like there's no tommorow. So hypocritical.
I thought people here were generally smart enough to know that security by obscurity doesn't work. Just because Joe Spammer doesn't care to tinker around to make his spam more devious doesn't mean Joe Hacker isn't gonna do it just for the hell of it and pass it along to Joe Spammer somehow.
Flame On...
All this is made possible by Microsoft's crappy security structure of their OS's.
You can't mine data, if you don't have access to the files that store that data.
As far as stopping the spam from coming in? We can do that. The methods for detecting spam in it's current state apply. Whether it's detecting Penis enlargment, phishing scams, XXX content...etc., we can already do that. So bring on the personalized spam I say. I can swat it away just as fast as if it didn't have your name on it.
Don't be so sure about that "modifying parameters" part. I sure see a lot of pink stuff with "Subject:" lines of "%SUBJECT" and so forth. Certainly doesn't lead you to doubt Rule #3 of the Rules of Spam.
Lacking <sarcasm> tags,
Sure, lets give the spammers some good ideas, how to go around the filters, how to personalize messages, and generally how to become more annoying.
Great idea! thanks for bringing this to light, I know how my inbow will look in a few days...
Emails like the article describe sound like identity theft. That sounds a lot more prosecutable than your average spam. I wonder if the average spammer would take the risk.
My other Slashdot ID is much lower.
That only works if the zombies aren't on a DUL [1].
Beyond that, it's pretty easy to spot zombies locally because they hit spamtrap addresses. Once they do, the sending IP gets locally blackholed on the spot without SMTP ever getting beyond "RCPT-TO"
[1] Dial-Up List: list of dynamic IP addresses, not always dialup.
Lacking <sarcasm> tags,
It's been tried. Microsoft won't support anything that doesn't ultimately give them control of all e-mail.
Beyond that, encryption or signing of the contents requires that the MTA accept the whole stinking pink pile before even considering routing it to /dev/null -- and then it has to burn a huge number of cycles doing the cryptography. That's a deal-breaker for serious mailservers, which handle mindboggling numbers of messages every second.
Lacking <sarcasm> tags,
Cognito ergo scum - I think therefore I spam.
:v)
Vik
My only problem with smtp-auth is that it represent a key validation mechanism and not a host validation mechanism. That is, one can assert that a sysadmin built a version of sendmail and generated a key for smtp-auth, but one cannot assert that a particular host *used* that key - only that a specific key was used in generating the authentication header. That's why I think it needs to be tied into a hardware level DRM or hostID mechanism.
As for delivery authentication, that's a another kettle of fish. Two systems point to point using host authentication could allow for a delivery authentication system though. It would simply be a matter of using each system's public keys to generate and exchange cryptographic signatures during envelope exchange. That is - one can verify delivery to a host using such a system, but not delivery to a user.
Excellent reply, BTW. Thanks a bunch,
--M
That's the point. They don't deal with policing bad behavior during envelope exchange. And the only way to do this is to verify hostIDs in order to track the system to its owner. In the end, the only system that can possibly work will be one that forces people to be legally responsible for the traffic sent from their systems, with an enforcement mechanism.
I agree. I'd be interested. If it works as well as you say then do as the romans do, wait...just post it.
When I have a kid, I want to put him in one of those strollers for twins and then run around the mall looking frantic.
I'm reminded of Mark Buller, the guy who improved the accidental enhancement mousepox into a 100% deadly disease even in mice vaccinated against it. A guy named Ramshaw was researching transmissable mouse contraceptives to deal with an overpopulation problem and spliced a gene for the immunosuppressant IL-4 into mousepox. Unfortunately, this led to the death of 60% of the test mice. Buller published research where he expanded on this idea by putting the IL-4 gene in a better spot and put in another gene to maximize production. This killed mice even treated with anti-viral drugs with a nearly 100% fatality rate.
Fortunately, however, Buller seems to have tried to make up for this a little by having come up with a counter-measure. This provides a hope for some people to live in case of genetically engineered smallpox, but I don't think that the kind of drugs required are even close to being common and inexpensive enough to help the public at large.
One of these days, I'm worried that unethical or thoughtless biologist are going to publish exploits for the human immune system, and one of these days technology is going to get cheap enough and ubiquitous enough for the biologist equivalent of a script kiddie to wage genocide. I'm worried that in the next century, we're going to get an object lesson in just how hard it is to "patch and update" our immune system.
If it's for-profit but free, you're not the customer -- you're the product (e.g., the Slashdot Beta's "audience").
That's really the question at the heart of all these smtp-auth schemes. At the hardware level you have an individual computer tied to a serial number and sales receipt. Once one can verify who owns the computer and that a message was sent from *that* computer and not some other computer, it then becomes possible for law enforcement to track down and stop specific systems from sending SPAM. It also becomes possible to track a variety of other illegal activities. Plus many legal ones.
Like I said, choose one: anonymity or protection from criminals. You can't have both.
I've already recieved one on the 9th April 2006 with my full name in the subject: "Important infomation to Mr Gavin {my surname} ." and also in the body:
" Dear Gavin {my surname} ,
I am Barrister Atiko Benson, a senior advocate,personal attorney to Mr.Andrew {my surname},who used to work with Shell Development Company in Lome Togo. Herein after shall be referred to as my client.
On the 21st of April 2001, my client, his wife and their only daughter were involved in a gastly car accident..."{continue classic nigerian scam}.
Talk about hitting the nail on the head. Who knew an 18-year-old needed Viagra?
Personalised spam assumes intelligence, something lacking on both ends of a successful spam message.
Do you see what I did there?
The TREC tests involved tests on 350,000 email messages. A 92,000 message public corpus from this effort is available for free download.
John Graham-Cumming (no relation to TREC) has created SpamOrHam -- a community-based effort to adjudicate the judgements in the TREC corpus. This'll let us test in a big way Yerazunis' contention that spam filters are better than humans.
Any filter writer can participtate in TREC 2006 by submitting a letter of intent now and a filter in due course.
There's also an upcoming scientific spam conference this summer - CEAS.
My stats right now:
Messages received: 9,466
Messages identified as spam: 394
Messages flagged with a virus: 1
Sure, it's possible to get better than that. But for the company I work for, the "spam problem" is effectively "solved".
And over time, it just going to get better as the spamtrap address I've been using are sold and re-sold amongst the spammers.
I'm sure others have even better stats. I'm using a mix of Exim4, greylisting and SpamAssassin along with my personal white/black lists (populated by the aforementioned spamtrap addresses).
As far as the load on mail servers, there's plenty of middle ground between waiting for an RFC or capitulating to DRM to fix the SMTP problem. Mindshare is the only real obstacle between the way things are & a least-privelige mail system that uses strongly signed logins integrating a sender/receiver pair hash. Hell, I'd use & spread an alternative and experimental system like that, standards be damned. I mean, where's the W3C spec for onion routers and torrents, et. al?
Pi Ran Out
If you can craft an email that uses a persons name and the city they live in, it's pretty obvious that the response rate is going to go up. Ultimately, we need spam shutdown at the SMTP servers. Since spam is free to send, it doesn't matter what the hit rate is, people are going to keep doing it.
Well, a very simple and easy way to smuggle cocaine into here, would be for the smugglers to create an underwater glider. The idea originated at either woods hole/Scripts, but they are operating a joint program on it.
Now how can a smuggler use this? A columbian drug lord can afford to research and create their own glider. It can then be loaded with several tons of coke. Yes, TONS. Then allow the glider to do its thing. How long will it take? Who cares as long as it gets there. It may have to move across the gulf or even from Mexico to Ca. The real issue here, is that it will not be spotted while under water except by divers. Since it is a glider, little to no noise. It can even be made out of aluminium or fiberglass.
Now, why is this of interest? Because, we talk about setting up a fence on the mexico border to keep out illegals and "terrorists". Now, you have seen a nice way to get a nuke or arms into America (I doubt that we have sensors in the middle of the gulf looking for radiation or the incredablly small amount of fiberglass that this would consists of). Basically, this info shows that here is one way to get ppl (iffy), drugs, arms, money for laundering, etc onto our shores. In fact, even simple gliders launched from mexico could do work (but much easier to spot).
And what good will the 7 Billion GWB fence as built by haliburton be? Absolutely NADA. So, yes, even thinking out some creative ways to ship drugs in here shows how porous we are.
Sadly, an even easier exercise is to read the history of USSR and China. For all their walls/fences and security, they have never had it.
I prefer the "u" in honour as it seems to be missing these days.
That's a value question. Do you value anonymity more than security from criminals? Each person must make their own choice, I suppose. There are many who would argue that the destruction of anonymity will badly impact the right of political dissent. Just as there are others who argue that the right of dissent is of less importance than a functional email channel. It's a thorny issue. At the moment I fall in with the law and order crowd, if only because email is so dysfunctional now that it is straining under the load from SPAM. Why worry about the right to dissent when the very functionality of the system is under threat?
Yes, that would be another solution. If a public/private keypair were automatically generated from the OS authentication system, and then the email client automatically handled say an X509 exchange, one could certify that a specific human being sent a specific message. However, that would still be open to abuse for those with root privs. Such a system would require implementation everywhere - though I suppose so would a DRM like hardware standard as well.
I agree with everything you wrote, other than the value judgment on whether a hardware or OS level encryption system is a necessity. Whatever happens, the current system is broken and cannot be fixed without some enforcement mechanism. Also, I would argue that encryption is just one piece of the solution, legal enforcement has to come next. IOW: throw the assholes in the slammer. --M
It amazes me that even though I have had some kind of personal website since 1994 with info on some of the things that interested me, I have never, in all of the years since then, received a single piece of spam that related to any of those subjects. You would think that with the capabilities of computers, spammers would make some effort to target their advertisements.
Can you think up an RFC that could solve the problem without some physical mechanism to track messages and owners of systems? I can't. This is not a purely technical problem, it is a problem of human behavior. For thousands of years laws and legal enforcement have been the only meaningful check against antisocial behavior. Can you write code (or propose an RFC protocol) that would do the same?
I've already run into several situations where email delivery was compromised due to spam / graylist filtering. IMO: it's a no-win arms race that can only lead to further dissolution of the communications channel. Attempting to filter out the noise only leads to filtering out some of the signal in the process.
Check out my sci-fi/humor trilogy at PatriotsBooks.
Except a better analogy would be if you were talking about breaking the DRM on every copy of a piece of software world-wide. Or, to phreak every phone in France (or wherever) to give free calls.
Both of your supposed analogies are actions against single instances of an inanimate object.
Perhaps you're trolling but, you know, spam scales to affect millions of people (if it works!).
I have actually been wishing spammers would do this for a long time.
Spam is not going to stop, so the best thing that could happen is they start to target people better.
br If Viagra spammers started sending their email only to men over the age of 40. A huge number of people would suddenly get a lot less spam
Never heard of SpamCop?
That's exactly what it's doing, plus checking links in the message to report spamvertised web sites as well.
Secure Historied Personae are the best proposal I've seen to sever the competing relationship between privacy & network security.
A network would grant nyms on a truly anonymous basis, but a newly acquired nym would only be as good as the door it comes knocking on decides it should be. That could be based on (still anonymous, but historied) individual encounters, or on what information the nym owner is willing to disclose in return for the right to access the resource in question.
Just like eBay accounts and /. karma, a nym reputation would take time & effort to build up, after which it'd be very much worth preserving. If the system is well secured against nym tracing *and* against forgeries of nyms or their histories, I think that'd represent a substantial improvement to both the privacy and the abuse prevention fronts.
http://en.wikipedia.org/wiki/Pseudonymity
Pseudonymity in the light of evidence-based trusta tions/Papers/spw04.pdf
http://www.cl.cam.ac.uk/Research/SRG/opera/public
Pi Ran Out
Spammers have more money than you do. They have more resources than you do. More bandwidth, more sending hosts, more CPU power. Now come up with a solution that works under those circumstances.
I solved my spam problem and talked about it here.
I was Slashdotted so my approach had some merit with some of the crowd here.
But in the end I was 'marginalized' for my efforts to make email useable again.
Anyway, you can read the Slashdot thread and visit my site if you want to for more information and to learn how I cut my spam received down to 1/3 of 1% (0.003333...)
Note: Due to the Slashdotting, the software is unavailable at the moment. I also have to tweak the software a bit to prevent it from hanging on bad, non-rfc compliant emails the spammers have sent me in the past. My site has helpful freeware/shareware there but it looks like the antispam freeware/shareware there got me 'shouted down' here for my efforts.
The spam problem has gotten worse and worse since I wrote the software. While everyone else buys expensive hardware email filters, use several 'complicated' pieces of software working together to fight spam, advocate a new email standard, or just bemoan the problem, I feel a genuine sense of accomplishment of fighting the problem at its most fundamental level and succeeding.
Thank you for your consideration.
Because I prefer to make things a little harder. There are actually quite a few firefalcon's out there, and similarly quite a few people with the same first and surname as me, so you may or may not be correct.
For the same reason, when posting logs to email lists, I tend to hide the IP address - even though it is very easy to find out.
I noticed that you also prefer a certain level of anonimity - posting as AC...