NYTimes: Tangled Up in Spam
ezekieldas writes "Congratulations to the SpamAssassin developers and community! There's a mention of SA in the NYTMag as "one of the best tools for network administrators..." in an extensive article entitled
Tangled Up in Spam.
The article is quite substantial and the author, James Gleick, is more technically educated than what we've come to expect from the big press. Central to the story is the complexity in dealing with spam effectively in both technical and legal terms and the confusion it brings upon the neophyte. The conclusion drawn may be oversimplified but nonetheless pragmatic: 1) forged headers should be illegal 2) a specific header entry should identify the email as unsolicited."
I been using Spam assassin for a while now, it is sad to say, but email would be almost unusable with out it.
now that it has been advertised in NYTmag, more people will become aware that spam is something they can actually stop. Can't wait for the new tricks spammers will use to disable anti-spam programs.
The conclusion drawn may be oversimplified but nonetheless pragmatic: 1) forged headers should be illegal 2) a specific header entry should identify the email as unsolicited
Why does everyone in the USA assume that everyone else in the world will somehow obey US law when it is made "illegal"?
now use SpamAssassin. Basically, a set of new headers is attached to the e-mail of the form X-Spam-foo, and if X-Spam-Score is 7.5 or greater (on a scale of 10 I believe), then X-Spam-Flag is yes. It's really useful for sorting out spam quickly, and I haven't gotten a false positive yet...It doesn't get all of the spam, but it gets the vast majority of it...
Spam is a technical problem, so why can't we come up with a technical solution? For example, it should be impossible to forge headers, not illegal. Why rely on a legal solution from many of the people who have brought us such brilliant solutions as the DMCA and the CDA in the past when all that's required is what our community has always been good at: sitting down and thinking things out?
Try not. Do or do not, there is no try.
-- Dr. Spock, stardate 2822-3.
Looks like we have the supremes on our side; if we could just congress to issue some letters of marque and reprisal on the spamhausen, we'd be getting somewhere...
What a strange bird is the pelican, his beak can hold more than his belly can.
I think that breaking that economic model -- ending the reciever-pays system for email -- is the only way to fix spam. If you had to pay some amount of money -- event 1 cent -- for each message that is delivered, spam would stop being economical. And that's the only thing that's going to make it stop.
-Esme
The one big feature missing for me in evolution is a spam filter. Fortunately, spamassassin works great even if you have to run it locally. Here are some instructions for evolution users who need to run it locally or are lucky enough to have spamassassin installed on their mail server.
I don't work for a large corporation, but a state-wide ISP. I asked my boss, the chief technical officer of the company, why we weren't using Spam Assassin. He replied that while it is a very neat program and does a great job of filtering spam, the performance just isn't quite there yet. He's of the mindset that it needs some tweaking still before it can be a competitor to commercial products like what Brightmail offers.
Personally, I'd like to see more companies using SpamAssassin just to prove that it can stack up against other products, because I think it can work well if it's configured properly and you use spamd. I use it on my mail server at home and at last check it catches 98.2% of all spam message sent to my machine, and I haven't had any false positives since I set up my whitelists.
Beef! Beef! Beef!
illegal is great in theory, but there is no possible way to enforce that on a world wide basis.
It's impossible to enforce almost any laws with 100% effectiveness, but that does mean that we should ignore the problem. If some sleazeball in Florida hires a firm in Korea to spam me, put his ass in jail.
white lists are the only way to stop spam.
I'm amazed by this user-hostile suggestion every time I hear it. Suppose you post your resumé on Monster.com. Who are you going to whitelist? Suppose your friend changes ISPs and then tries to e-mail you his new address? It won't be whitelisted, so it will bounce. Suppose to fill out a tech support request form. You don't know the address of the person that will contact you (or even if they will be the same domain as the web site).
I've been using Cloudmark's SpamNet for the past few months and it's been working quite well.
The smart thing that SpamNet does, is that it relies on its users to determine if something is spam or not. If some email lands in your inbox and a few hundred SpamNet members have proclaimed it spam, it most likely is, and it gets immediatly filtered out. This has the net effect of a few user's needing to filter out a few message ocassionally, while the vast majority of messages are filtered out for all users. Although SpamAssassin seems quite good, it's still based upon filtering rules and spammers are constantly tweaking their emails to try to get around them. Since people are still better at determining what's spam and what's not, I find that its accuracy is generally better.
SpamNet isn't perfect though, as far as I know, it only works with Outlook on Windows and doesn't have a Unix, Linux or Mac version. It also sometimes filters out valid bulk mailings, but overall, I would definitely recommend it.
SpamAssassin's a great idea, but for the non-technically minded user, POPFile's the best choice. Bayesian filters, learning, kickass UI, and a Windows installer (and Perl for other platforms.)
>>1) forged headers should be illegal 2) a specific header entry should identify the email as unsolicited
Don't we ever learn from the past? We've all seen the unintended consequences of poorly-crafted legislation (e.g. DMCA), so why run to the shelter of more restrictions which, in the end, will only cause us more problems? Like the criminals trying to scam your mom with the Nigerian-hold-my-money-for-a-day scam are going to suddenly begin obeying the law... yeah, right. Which begs another question: what law, in what jurisdiction? Even if the US were to pass this law and ruthlessly enforce it (domestically), all scammers would simple flood us from offshore servers.
The solution is not legislation, it is the creative use of technology. Build software that "learns" what is spam and what isn't, then evolves to keep up with the changing tactics of the spammers. Something like PopFile
1. Spend 10 bucks, buy a domain name (eg xyz.com).
2. Set up a few email aliases to point to your real email. eg:
joe@xyz.com ---> you@hotmail.com
temp123@xyz.com ---> you@hotmail.com
spam123@xyz.com ---> you@hotmail.com 3. Never give out 'joe@xyz.com' to anyone except friends/family.
4. Use the other emails for signing up for things on the web or in usenet.
5. When you get your first spam addressed to 'temporary21@xyz.com', delete the email address (no more spam from that source!).
I find this method works extremely well. By using aliases in this way you effectively hide your real mailbox. Even if your hotmail account starts receiving spam you can just get a new one and point your aliases at it. Also, if you change ISP you don't need to change your email address.
If you use it to forward to a hotmail account it might be better if the hotmail account name isn't a dictionary word or name (ie. use a random string for an account name that the 'bots won't guess.
You're screwed if your 'trusted' address gets out there but if you're careful you'll at least get much more use out of it before needing to kill it.
For what it's worth, an ever-so-slightly longer version, lacking a few bits of Times editing, is posted here, at my own site. And may I say how helpful and fascinating the many Slashdot discussions of this subject have been?
If we can pull it off.
With Bind 9, we finally have a decent, working implementation of DNSSEC. This will allow for a new breed of secure, verified websites and email, and (Finally!) makes a RBL actually mean something.
How's that you ask?
Well, one of the biggest problems with SPAM is the forged header, open relay issue. It's a complicated issue, and one that doesn't have an obvious, "in your face" kind of answer.
DNS is designed to tell you where to go, and SSL/Certs make sure that you got there. Why aren't they joined together? The fact that you are the DNS server for a domain makes it clear and obvious that you are an authoritative designator for where you are supposed to go - why have this wholy separate and dis-jointed SSL/Cert that can't even be made to work consistently?
If an ISP can issue DNS-SEC certs with impunity, we might actually see a reason to have encrypted and ISP certified email.
And suddenly, the ISP is back in charge again, able to validate every email going out as coming from one of it's customers. Revoke the cert and their email becomes unreadable.
Now, we have an email system with a powerful mechanism built in that is:
1) Standards compliant
2) Easy to implement
3) Clearly laid out
4) Cheap
5) secure
6) private - using the ISP's cert to identify yourself doesn't mean that the ISP can read your email! (like they can now - the command is "mail -u _username_")
What's not to argue with? The issue of locking down an open relay becomes a non-issue - an ISP could simply identify an "s-mail" server (secure mail) that will only relay for those holding a valid cert at that ISP.
Roaming wouldn't be an issue, nor would open relays or forged headers.
A brave new world? Yep. One I'd like to live in? Yep. One that's coming? We can only hope...
I have no problem with your religion until you decide it's reason to deprive others of the truth.
>URGENT ASSISTANCE - FROM USA :
>
>IMMEDIATE ATTENTION NEEDED
>HIGHLY CONFIDENTIAL
>
>FROM: GEORGE WALKER BUSH
>202.456.1414 / 202.456.1111
>FAX: 202.456.2461
>
>DEAR SIR / MADAM,
>
>I AM GEORGE WALKER BUSH, SON OF THE FORMER PRESIDENT OF THE UNITED STATES
>OF
>AMERICA GEORGE HERBERT WALKER BUSH, AND CURRENTLY SERVING AS PRESIDENT OF
>THE UNITED STATES OF AMERICA. THIS LETTER MIGHT SURPRISE YOU BECAUSE WE
>HAVE NOT MET NEITHER IN PERSON NOR BY CORRESPONDENCE. I CAME TO KNOW OF YOU
>IN MY SEARCH FOR A RELIABLE AND REPUTABLE PERSON TO HANDLE A VERY
>CONFIDENTIAL BUSINESS TRANSACTION, WHICH INVOLVES THE TRANSFER OF A HUGE
>SUM
>OF MONEY TO AN ACCOUNT REQUIRING MAXIMUM CONFIDENCE.
>
>I AM WRITING YOU IN ABSOLUTE CONFIDENCE PRIMARILY TO SEEK YOUR ASSISTANCE
>IN
>ACQUIRING OIL FUNDS THAT ARE PRESENTLY TRAPPED IN THE REPUBLIC OF IRAQ. MY
>PARTNERS AND I SOLICIT YOUR ASSISTANCE IN COMPLETING A TRANSACTION BEGUN BY
>MY FATHER, WHO HAS LONG BEEN ACTIVELY ENGAGED IN THE EXTRACTION OF
>PETROLEUM
>IN THE UNITED STATES OF AMERICA, AND BRAVELY SERVED HIS COUNTRY AS DIRECTOR
>OF THE UNITED STATES CENTRAL INTELLIGENCE AGENCY (CIA).
>
>IN THE DECADE OF THE NINETEEN-EIGHTIES, MY FATHER, THEN VICE-PRESIDENT OF
>THE UNITED STATES OF AMERICA, SOUGHT TO WORK WITH THE GOOD OFFICES OF THE
>RESIDENT OF THE REPUBLIC OF IRAQ TO REGAIN LOST OIL REVENUE SOURCES IN THE
>NEIGHBORING ISLAMIC REPUBLIC OF IRAN. THIS UNSUCCESSFUL VENTURE WAS SOON
>FOLLOWED BY A FALLING-OUT WITH HIS IRAQI PARTNER, WHO SOUGHT TO ACQUIRE
>ADDITIONAL OIL REVENUE SOURCES IN THE NEIGHBORING EMIRATE OF KUWAIT, A
>WHOLLY-OWNED U.S.-BRITISH SUBSIDIARY.
>
>MY FATHER RE-SECURED THE PETROLEUM ASSETS OF KUWAIT IN 1991 AT A COST OF
>SIXTY-ONE BILLION U.S. DOLLARS ($61,000,000,000). OUT OF THAT COST,
>THIRTY-SIX BILLION DOLLARS ($36,000,000,000) WERE SUPPLIED BY HIS PARTNERS
>IN THE KINGDOM OF SAUDI ARABIA AND OTHER PERSIAN GULF MONARCHIES, AND
>SIXTEEN BILLION DOLLARS ($16,000,000,000) BY GERMAN AND JAPANESE PARTNERS.
>BUT MY FATHER'S FORMER IRAQI BUSINESS PARTNER REMAINED IN CONTROL OF THE
>REPUBLIC OF IRAQ AND ITS PETROLEUM
>RESERVES.
>
>MY FAMILY IS CALLING FOR YOUR URGENT ASSISTANCE IN FUNDING THE REMOVAL OF
>THE PRESIDENT OF THE REPUBLIC OF IRAQ AND ACQUIRING THE PETROLEUM ASSETS OF
>HIS COUNTRY, AS COMPENSATION FOR THE COSTS OF REMOVING HIM FROM POWER.
>UNFORTUNATELY, OUR PARTNERS FROM 1991 ARE NOT WILLING TO SHOULDER THE
>BURDEN
>OF THIS NEW VENTURE, WHICH IN ITS UPCOMING PHASE MAY COST THE SUM OF 100
>BILLION TO 200 BILLION DOLLARS ($100,000,000,000 - $200,000,000,000), BOTH
>IN THE INITIAL ACQUISITION AND IN LONG-TERM MANAGEMENT.
>
>WITHOUT THE FUNDS FROM OUR 1991 PARTNERS, WE WOULD NOT BE ABLE TO ACQUIRE
>THE OIL REVENUE TRAPPED WITHIN IRAQ. THAT IS WHY MY FAMILY AND OUR
>COLLEAGUES ARE URGENTLY SEEKING YOUR GRACIOUS ASSISTANCE. OUR
>DISTINGUISHED
>COLLEAGUES IN THIS BUSINESS TRANSACTION INCLUDE THE SITTING VICE-PRESIDENT
>OF THE UNITED STATES OF AMERICA, RICHARD CHENEY, WHO IS AN ORIGINAL PARTNER
>IN THE IRAQ VENTURE AND FORMER HEAD OF THE HALLIBURTON OIL COMPANY, AND
>CONDOLEEZA RICE, WHOSE PROFESSIONAL DEDICATION TO THE VENTURE WAS
>DEMONSTRATED IN THE NAMING OF A CHEVRON OIL TANKER AFTER HER.
>
>I WOULD BESEECH YOU TO TRANSFER A SUM EQUALING TEN TO TWENTY-FIVE PERCENT
>(10-25 %) OF YOUR YEARLY INCOME TO OUR ACCOUNT TO AID IN THIS IMPORTANT
>VENTURE. THE INTERNAL REVENUE SERVICE OF THE UNITED STATES OF AMERICA WILL
>FUNCTION AS OUR TRUSTED INTERMEDIARY. I PROPOSE THAT YOU MAKE THIS
>TRANSFER
>BEFORE THE FIFTEENTH (15TH) OF THE MONTH OF APRIL.
>
>I KNOW THAT A TRANSACTION OF THIS MAGNITUDE WOULD MAKE ANYONE APPREHENSIVE
>AND WORRIED. BUT I AM ASSURING YOU THAT ALL WILL BE WELL AT THE END OF THE
>DAY. A BOLD STEP TAKEN SHALL NOT BE REGRETTED, I ASSURE YOU. PLEASE DO BE
>INFORMED THAT THIS BUSINESS TRANSACTION IS 100% LEGAL. IF YOU DO NOT WISH
>TO CO-OPERATE IN THIS TRANSACTION, PLEASE CONTACT OUR INTERMEDIARY
>REPRESENTATIVES TO FURTHER DISCUSS THE MATTER.
>
>I PRAY THAT YOU UNDERSTAND OUR PLIGHT. MY FAMILY AND OUR COLLEAGUES WILL
>BE
>FOREVER GRATEFUL. PLEASE REPLY IN STRICT CONFIDENCE TO THE CONTACT NUMBERS
>BELOW.
>
>SINCERELY WITH WARM REGARDS,
>
>GEORGE WALKER BUSH