Slashdot Mirror
NYTimes: Tangled Up in Spam
ezekieldas writes "Congratulations to the SpamAssassin developers and community! There's a mention of SA in the NYTMag as "one of the best tools for network administrators..." in an extensive article entitled
Tangled Up in Spam.
The article is quite substantial and the author, James Gleick, is more technically educated than what we've come to expect from the big press. Central to the story is the complexity in dealing with spam effectively in both technical and legal terms and the confusion it brings upon the neophyte. The conclusion drawn may be oversimplified but nonetheless pragmatic: 1) forged headers should be illegal 2) a specific header entry should identify the email as unsolicited."
lol, that looks exactly like something exploiting a well-known but still unpatched Windows vulnerability. :-)
The one big feature missing for me in evolution is a spam filter. Fortunately, spamassassin works great even if you have to run it locally. Here are some instructions for evolution users who need to run it locally or are lucky enough to have spamassassin installed on their mail server.
when people say SpamAssassin is good - they should really be talking about 2.5
that is the version with the Bayes fully in it and it is head and shoulders above the previous versions IMO
There are some odd things afoot now, in the Villa Straylight.
The uneducated guy that send this story in, need to know that was instrumental in taking Chaos theory from an obscure science in Santa Fe into something that almost every scientific discipline benefits from. Incl CS. .
Help fight continental drift.
illegal is great in theory, but there is no possible way to enforce that on a world wide basis.
It's impossible to enforce almost any laws with 100% effectiveness, but that does mean that we should ignore the problem. If some sleazeball in Florida hires a firm in Korea to spam me, put his ass in jail.
white lists are the only way to stop spam.
I'm amazed by this user-hostile suggestion every time I hear it. Suppose you post your resumé on Monster.com. Who are you going to whitelist? Suppose your friend changes ISPs and then tries to e-mail you his new address? It won't be whitelisted, so it will bounce. Suppose to fill out a tech support request form. You don't know the address of the person that will contact you (or even if they will be the same domain as the web site).
Some more clarification:
-it's not on a scale of 10 - the SA score can go as high as necessary. I got 27 the other day. Your threshold will be configurable (sometime next week) to "high" (3.0), "normal" (7.5), or "low" (12.0), or a custom number. You'll also have custom whitelists and blacklists.
There is no sig, there is only Zuul.
SpamAssassin's a great idea, but for the non-technically minded user, POPFile's the best choice. Bayesian filters, learning, kickass UI, and a Windows installer (and Perl for other platforms.)
Go figure.
Is this thing on? Hello?
The most important Q, if gov't help is going to mean anything.
Enforcement is currently a state problem, for the dozen or so states that have antispam laws. Even if they can establish jurisdiction, they have to locate the offender. An asst. attorney general I chatted with in Washington state described an almost comic crusade to get ONE spammer who set up under a different corporate name every week. They used three private investigators to track him (successfully), suggesting to me their investigatory resources were limited. Anyway, they couldn't afford to do this with everyone, and this one example was located in-state!
I was surprised the author didn't really talk about state laws at all. They're kind of the laboratories for the eventual federal effort, and state law/enforcement will be complementary.
Once there is a law on the books the "cyber" aspect of it is only as issue for tracking. Postal mail and telephone calls have "no physical boundaries," too, and actually it is the crossing of state lines taht is an obvious source of federal jurisdiction. The rest is standard law enforcement. The FTC, which the author briefly visited, was busy enough with outright fraud, where it already has jurisdiction, just as it does over fraudulent TV ads and newspaper ads and product labeling and so on. I can say that I've seen some very good work by the FTC, even leading to jail terms for the guys who just won't give up. (The jail term I saw was for criminal contempt of court.)
I think they're going to need to provide a private enforcement action, as with the fax law. The gov't resources would still be needed to track down and prosecute the really tough ones, such as the WA case I described. We already have some relevant experience from the anti-junk fax law.
Recognizing spam -- good Q. I don't have any trouble recognizing 99% of it. For teh false positives, it should be possibly to allow the merchant to provide evidence of opt-in, and if enough complaints are tallied there would be further action.
Yes, deploying a more trusted protocol will take several years to reach every corner of the Internet. Sounds like a good reason to start immedately.
The reason this could work is that the Internet is not as decentralized as you make it out to be. Between MSN/Hotmail, AOL/Netscape, Yahoo, Earthlink and the telcos/cablecos, you've got about 90% of personal mail accounts.
Your company will upgrade when they find that your salesdroids get less favored status at Hotmail, etc. Any larger company has the staff to go upgrade sendmail.
Most smaller companies have upstream ISPs that could relay from legacy SMTP to the secure version, or they just use the ISP relay directly.
It will take a while for the Chinese and the Koreans, etc but many sites block those domains entirely.
After a year or two, the big sites just shut off normal SMTP entirely. All the stragglers will get the right idea quickly.
If you just want a fake email address that is "valid", use whatever@example.com
example.com is an official internet blackhole, sanctioned by RFC. It is what everyone is supposed to use in books, demonstrations, etc, similar to 555-XXXX phone numbers on TV.
Better strategy.... But requires having control of your own mail server...
.forward-amazon and have it put mail in /dev/null. Alternatively you could use procmail or maildrop in the dot-forward file to perform per-extension filtering or bounce messages to explain why the mail will never be read, in case legitimate mail tries to come into that box, perhaps with a random, unique extension provided for them to try a legitimate box. Not only do you have an effective mechanism for filtering out unwanted mail by source and outdated email, you also have a way to track how your email gets out. It has worked quite well. Last week I got three spams, and blocked that address. Aside from that and a couple of other incidents in the past year (about 8 or 9 spam mails total), the signal to noise ratio in that mailbox is excellent.
I run my own mail server. I have Postfix configured to forward username-@the.server to username@the.server by default. So, for example, I registered with amazon username-amazon, and it gets to me. If this email is ever put on a list, I'll complain to amazon, and then create a
XML is like violence. If it doesn't solve the problem, use more.
I'm less psyched about filtering at the router (mail server). Two words: arms race.
Having each mail server filter on content along the chain would work in the short run, as soon as it became too effective, the spamers would think of ways to eeek by the ratio. Lower the ratio, so would the spammers till you end up filtering out mail that is legitimate.
(That, and I'd hate to have to spec a system that would do that filtering without adding substantial delays!)
Beyond adding a cone of paths like you first described, and figuring out other technical ways to deal with this, I see a couple things that will probably be required in the future;
(Ob comment: Yes this is a big deal, involves pain, is likely not backward compatable, and should be thought out very carefully.)
A firewall can not protect you from yourself. Turn off what you do not need. Do not use the firewall to do your work.
Consider the following. We all access the internet from a fixed and typicaly small number of physical and virtual locations. Were we to map the internet as a whole, starting from any given location the map would look like an expanding cone.
Actually, it wouldn't due to the multihomed nature of most networks.
Since spam messages are sent by the millions and it is fairly easy to determine what messages are likely to BE spam why not set up a filtering system on the routers that determines the rough content of a message based on both its Spam Precentage and the number of identical messages sent.
I.E. If the router sees 500,000 messages of nearly identical content with a 89% spam rating it blocks all of them. If it sees 44 messages with a 23% spam content it lets them through.
First, routers are meant to do one thing, route traffic. They do not have the memory or CPU power to do much more than that.
Second, "identical" and "near-identical" messages are very different things. It is fairly cheap (processor/memory wise) to determine if two messages are identical. It is quite another task to determine if they are nearly identical.
Third, there are many instances where identical or nearly identical messages sent out in bulk are not spam. Mailing lists like bugtraq or linux-kernel have very large subscriber lists, but are are not spam. If the head of IBM sends a message to all his employees, it is not spam. If my car insurance company sends out a bunch of messages warning people once a month that their policy will expire if payment isn't received, it is most definitely, not spam.
The world is neither black nor white nor good nor evil, only many shades of CowboyNeal.
For what it's worth, an ever-so-slightly longer version, lacking a few bits of Times editing, is posted here, at my own site. And may I say how helpful and fascinating the many Slashdot discussions of this subject have been?
If you use MS Outlook (we are forced to at work), try out Spammunition. It's a free Bayesian spam filter that's integrated right into Outlook. Works really well. No spam problems any more. This bayesian approach really works.
Spamassassin has various tests for this type of behavior. e.g.
Message text disguised using base-64 encoding BASE64_ENC_TEXT
However with the current default scores that alone would not flag a message as spam.
BalamHeh. I assume you are honestly asking and not bragging about how little SPAM you get to make me jealous...
Here are the vectors for getting on lists that I know of;
- using a valid email address in newsgroups
- using a valid email address on a web page
- using a valid email address in form properties in a web page
- using a valid email address on a mailing list or web-forum
- using a valid email address for domain registration contacts
- using a valid email address to sign a web page up for a search spider
- having an email address that can be "brute forced" (i.e. almost all of them)
- your pal puts an email address in an "e-vite" or "e-greeting"
- getting a virus that spreads via email
And above all, being naive about the workings of the Internet, when only a few weeks of ignorance will permenently get the address out there "in the wild". Just about everybody is this at one at one time or another.
Some people cannot avoid having email addresses hung out there on the Internet, so getting on the lists is more or less inevitable if you are doing business or communicating on the Internet in any meaningful way. Since I cannot ignore what comes in the boxes I run, I MUST sort through whatever arrives. That makes SPAM a big issue for me.
Your usage of your email addresses is probably typical (not on web pages and so on..) but you are probably fortunate to both be clueful about it and not dealing with your email address publicly available out of necessity.
"the author, James Gleick, is more technically educated than what we've come to expect from the big press."
Maybe because after many years as a reporter, he founded Pipeline, one of the first big ISPs.
Tell me about it. I deal with that a lot. I mean, look at my email address. It's nice to have a simple one like that, but the amount of spam I get is ridiculous. 100+ a day. I also strongly suspect a particularly bitter ex-girlfriend of signing me up for all sorts of crap. I know she got my email into initial circulation with those damn "Someone's got a crush on you" crap. That's about when I started getting unreasonable spam, about 2 years ago.
On the bright side, OS X's Mail.app has an extraordinary spam filter. Very few false positives (about 2 in a couple months, I think). The occasional spam slips through, but only a couple a week. Considering the amount I get, it's been a great relief.
And to all you damn spammers out there, I don't know who the hell "JOE BLACK" is, unless you think I bear a strinking resemblance to Brad Pitt. In which case, thanks for the flattery, but fuck off.
COMPUTER! Whatever happened to Blueberry Muffin?
Can't wait for the new tricks spammers will use to disable anti-spam programs.
Wait no more. I got a spam today that purported to be an apology for how the sender got my address, something like "so sorry, but these stupid porn sites like [link] must have sent me a virus. I can't believe my kids are visiting sites like [another link] even though I never go to sites like [yet another link], blah blah blah."
I have to admire the creativity of spammers even as I wish for Bad Things to happen to them.