Slashdot Mirror
Analysis of Spam, and a Proposed Solution
2bot_or_not_2bot writes "Spam: The Phenomenon is a detailed analysis of spam: products, scams, viruses, obfuscation methods, etc. Failed, and doomed-to-fail, methods of blocking spam are described. A general solution is proposed that does not: invade privacy, perform wide censorship or blacklisting, or involve payment and cooperation with corporations (beyond the transport and storage of data)." Hmmm.
i don't know about any IT people around here but the biggest problem that I've been facing is getting back control of Hi-jacked computers. The tools out there to fix the problem just don't cut it 3000 search bars, start page hijacking, related pop-ups, malware, programs that just wont un-install. Its bad enough that they install in the background but there should be a "law" to make programs uninstall-able. Also make them from hiding there presence.
Counter Spam Measure: Negative Feedback.
Imagine if all or some very large contingent of email clients allowed you to
"retaliate" against spam messages. Highlight message, select "negative feedback"
option, a daemon is spun that traces back as far as possible the route of the
message and barrages it some fashion. By pings maybe? By directed replies? Imagine
it does this in some scheduled fashion so as to minimize the impact on your local
network. As 1 million disparate sources converge upon the last traceable source of
the route of the offending spammer, some network somewhere will start to feel the
load. Like the spokes of a wheel converging on the hub, the retaliation traffic will
thicken as it closes in on the source. The pain increases. ISPs inundated by
individuals expressing their right to freedom of speech, will feel suddenly inclined
to exercise their right to refuse service to someone.
The "negative feedback" could be dosed in a coordinated fashion if there were some
P2P means of establishing how many individuals had received a particular spam. If a
spammer hits only a hundred people, the dose of retaliatory traffic would have to be
increased to be felt. If the spam hit a million, it would require only a modest
retaliation to utterly swamp the source.
Just thinking out loud. Could this be made to work? No one's free speech is
curtailed, spam is dealt a serious blow.
fight fire with fire.
I have always wondered if the following solution would work: Say you wanted to send an email to your friend Sam. You would then put the word Sam as the very first word in the subject line. When the email is sent, Sam's email program verifies that the first word in the subject is Sam (Or any other word Sam chooses). If this is not the case, the email is blocked. Since 90% of the time, you are sending an email to someone who's first name you know, this might work. As for company email addresses, maybe just the company name or some special name would work. Since only the first word in the subject would be checked, spammers would have a very slim chance to guess the right name and get the email through.
Personally I rally liked D. J. Bernstein's (qmail, djbdns, daemontools) idea for a new mail protocol. The big difference between it and mail we have now is that only the notification of mail is sent, not the mail itself. The mail sits on the senders mailserver, waiting to be picked up, and if you want to retrieve it, your mail client does so from his server. Think about it - No more anonymous spam, since you KNOW where messages are coming from if you have to retreive them. Therefore, if spam is illegal, we can punish them... and there is no more faking of where its coming from.
The other cool concept to that is mailing lists vs bandwidth. In old mailing list styles, a message would go out to the list, bouncing back from all people whos boxes are gone or full- witha lot of traffic. In DJs new way, there is only notification of the message sent, and then only those who really want the message download it.
The more you think about it, the better of an idea it becomes. In the wold of terrifying ideas like "postage for emails" or "really super-mega-expensive domain names for mail only" Bernsteins has an elegance and practicality I haven't seen elsewhere.
I've been swashdotted -- Elmer Fudd
This worked great for me up until a month or so ago.
:(
My business partner's wife emailed me a 'get free movie tickets, just get five friends to sign up' email. I immediately dumped it into the trash, but apparently the fact that she had submitted my email to this place was all I needed to start the spam floodgates.
It still isn't too bad, but I am now getting 3 - 5 unsolicited SPAMs to that address daily from various companies. No p3n1s enl@rgements yet, but I'm sure that is just a matter of time.
This just bolsters my contention that people should be given a basic intelligence test before they are let loose on the internet. My partner's wife would certainly fail.
It's not my fault! It was this way when I got here.
Seriously? Go to a syn-syn/ack-ack system.
The sending SMTP box says to the receiver "I've got a message for you" Receiver caches the message, hands the source box a 32 digit random number and says I'll call back in 30 seconds by your FQDN. It does so. Receiver says "did you send me a message with the serial 'x'"? If yes, then the source in the header wasn't spoofed, and the message goes through, if not, the message gets dropped.
Almost all spam these days comes from spoofed sources. But if in this case it's still spam, it's a lot easier to track the source immediately and deal with it. Take away the ability to hide, and like mold in the sunlight, most of it will vanish without further effort.
Too bad nobody is combining those with a SMTP engine that can see the messages comming in and accept them VERY SLOWLY. (ie.: 1 byte per second)
I know something like this exists already but why not make known spammer servers get 3rd rate service from our owns servers?
Their servers would could take weeks to send out the number of messages they can now in 10 minutes. They need to get the massages out quickly or else the ratio of misses starts to cost them. This is the real solution.
Obama's legacy: (N)othing (S)ecure (A)nywhere and (T)error (S)imulation (A)dministration
This 'article' dismisses laws outright. Sure, bad laws, like in the USA, haven't worked. But look at europe! Successful laws, minimal spam.
It never ceases to amaze me what crap articles get accepted while quality ones get rejected.
That way you can use different addresses for mailing lists, orkut, random recipients, each Slashdot posting, etc., and blacklist addresses that get abused and/or only whitelist addresses you've sent people. There are some risks - the subdomain version occasionally gets hit by dictionary attacks, so you might receive 10 million messages on an occasional really bad day (this mainly happens if your subdomain doesn't run its own SMTP server that can milter it.)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
This is simple and requires no changes to a mail client to function, but one small change would make things easier. The solution does not need to happen all at once to be effective, and does not change any of the current protocols for email (POP,IMAP, SMTP).
The idea: multiple, sender/use specific addresses on the client side. Basically instead of having one address with your ISP, you would have the ability to create up to 50 aliases to your account. Not that these are not 50 accounts, all of your mail still winds up in the main mail account at your ISP.
Lets say you have bob.smith@myisp.com as your email address. The goal here is that you would NEVER give out that address. Instead, you log in to your ISP's web site and create addresses that you then give out. These addresses can be set to expire after a set date, or only be removed manually.
So you like to pay your bills on-line, create an address bobsbilling@myisp.com and use that on all the registration forms for your utilites, credit cards, etc.
bobs-shopping: use it to register for any on-line shopping sites
bobs-long-ebay-address, sendmailtobob, tossaway32341, etc....
You create an address that you give only to your family/friends, you create an address for each mailing list, create an address that you put in the public LDAP systems and other person-search sites, create an address for sweepstakes/contests, etc.
If you start to get spam on an address (you can easily check the headers to see which address the spam was sent to), you simply change the address and tell the few people/sites that used that address about the new one. The more addresses you have, the fewer places you need to notify of any changes.
The only disadvantage is the initial changeover does take some time/effort. Once created, the addresses mostly just sit there and don't require any maintenance or routine changing.
The advantages: little to no spam; abliity to easily identify WHERE the spammer acquired your address when you do get any; spam does not take up any bandwidth or storage space on the recieving mail server once an address is deleted after getting spammed; no resource intensive and complicated filter software required on the server.
How well does it work? With about 35 addresses out there (may are web site specific), I receive only about 6 spam messages a month. Each and every one of those is sent to a public administrator address like webmaster, hostmaster or the like, not too bad considering I recieve such email for about 10 domains.
In the last year or so since I've started doing this I have only had to disable a single address due to spam, and since it was for a single web site, it took less that five minutes to effect the changeover to a new address.
To those who say that this is too much of a hassle or takes too much effort, I ask this: would you rather have to spend 30 minutes a year maintaining and changing email addresses and informing senders of the new address, or spend 5 minutes a day updating your spam filters and double-cheking the positive results for false hits?
As I stated, this does not require and changes to the mail clients, but if there were one change it would be nice: when you reply to a message the client should automatically use the address that the initial message was sent to instead of attempting to use the actual account address.
Article X: The powers not delegated... by the Constitution...are reserved...to the people
Interactive filtering of SpAm by targets/users is best.
... maybe a couple other protocol/apps to provide identification and routing within TCP/IP packets for login, email, web-surf, VoIP, ... so many check, verify, route, ....
...), a group of addresses (job change, organization name update, ...), or all addresses (global list update/upload, reduce complexity, dropout, ...).
....
...) added to a user-AEL, or enough URL information to link back to an online business/interest website to track resent online banking, trading/investing, purchases, subscriptions, ... print invoices, or ....
I think; maybe, valid personal email should be the focus.
We want our email, but we do NOT want sPaM.
Currently we use USRID/AccID, DNS, DHCP, ARP-RARP,
I agree, with others, the W3C (someone) will need to add some RFCs on check/verify local "Lookup" user approved filter for email.
As Relates to SpaM/Email:
1. Subscribers, customers, users of an email service must be required to define an "Approved Email List (AEL)". Email client applications should require a user-action (right-click-select option, maybe) to generate a UDP/TCP update-message to add an addressor's email to the user-AEL resident on the email/profile server. To delete any addressors from a user-AEL should require a few extra steps of accessing the user-account web-page and specifically selecting one address (we change friends, someone moves,
2. Email service providers must provide to users a web-app/text-upload process for managing a user-AEL. (1) Either upload formatted text (with total content overwrite option) user-AEL as part of the user account/profile definition, or (2) on the email service domain's open/manage email account website a web-app that allows easy addition/deletion to the user-AEL.
3. New/Unknown email addressors, those not identified in an addressee user-AEL, with a datagram over 128-bytes (standardized size more/less for one name and an email address) are terminated, not delivered, bit-bucket, not replied/forwarded,
4. New/Unknown email addressors, those not identified in an addressee user-AEL, with a datagram under 128-bytes are delivered to the email addressee. This will allow the email addressee their option to decide; if the email addressor should be added to their user-AEL. This will allow an addressor to provide enough information to be potentially (as family, friend, business, hobby,
5. Incoming email are checked for valid local email accounts (NOT, then terminate). Incoming email having a valid local address are then checked by comparing the addresses with the user-AEL with the specific email address (userid@domain.___) of origin (MATCH NOT, then terminate). Repeat email terminations/rejects from same "@domain.___" could be blacklisted as a sPam@domain.___ unless recognized by a local user-AEL.
I'll stop counting here, because I think the rest can be surmised and counting gets boring. This process could be close to transparent for email users, except for the managing of an email account user-AEL. It would reduce spAM and potentially malicious/viral email in obvious ways by limiting allowed payloads/datagrams from unknown (un-validated/vouched for) sources in any email. Vouched for addressors (causing problems) on a user-AEL could be more traceable. The processing/handling overhead of such a systems would (I expect) be about the same as the present process and would significantly reduce email-server storage space requirements. Email is un-trustable, but required tool in the business world, and increasingly burdensome of our personal time.
The spAm-cans could only dump to email users that included them in their user-AEL. Over time it would reduce the spam-flood and/or spam-DDOS on the internet, because few (maybe none) would ever see spam-stuff and SPAM would prove a financi
Unaccountable leaders are masters, and unrepresented people are slaves. How do US and EU fare?
I imagine the problem is upgrading all those servers, or coming up with a transitionary system that allows both to exist (via trusted gateways?).
True, but if Sendmail and all of the other big mail packages got together and agreed on a date to have the upgrades available and working and then released the update packages on/by that date, you could have this auth as a switch to turn on at each SMTP server. Then when the implementation date passes, a lot of the big sites like AOL, Hotmail, etc. get it going, and if your company/ISP doesn't do so as well, you can't send mail to those folks anymore.
I remember the days when open relays were the norm and then there was the big push to close them. Our company got on the RBL and couldn't send mail. That got our ass in gear to fix it right away, and nobody died. This would be much the same, methinks.
I have a similar situation, an address I've had a good 15 years and it's so swamped with spam I'm regretfully coming to the conclusion it's not worth having anymore. But, if I only had control of the mail server...
I've got a much simpler method of stopping spam, and my analysis of the spam I receive tells me it would kill the vast majority of it. The author of the article almost mentions it, but discards it, wrongfully I think. He says
But he's wrong. I don't think I've ever once gotten a legitimate email in HTML. Trouble is it's no good to download the damn things before I can see that they're HTML, for it to be an effective remedy it needs to be implemented on the server. I think if email clients quit interpreting HTML (which they never should have done to begin with) or servers started simply refusing to accept messages in HTML, SPAM would, if not totally die, be dealt an incredibly powerful blow.=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
Step 1: Salt the spammer's email databases with guaranteed bogus email addresses that no legitimate email sender has ever seen. This is currently trivially implemented as follows. In your website's robots.txt file, list several files that robots must not examine -- these are your honeypot. Then, fill those files with HTML that contains your bogus email addresses. Spammers will, quite reliably, disobey the robots.txt file, use it to discover HTML files that are not linked to from anywhere else in the world, and add your bogus mail addresses to their database.
Step 2: Implement greylisting + honeypot-based RBL. When email arrives that is not whitelisted, see if it comes from an IP address that is "temporarily" blacklisted in your RBL. If it is, you can reject it right now. Otherwise, see if the target address is in your honeypot database. If it is, add the sender's IP address to your RBL and fail immediately. Otherwise, engage the now-classic greylisting algorithm (see http://www.greylisting.org/) to "tempfail" the email. The point of the temporary failure is to give the spammer time to use the same IP address to send the same spam to an address that *is* in your honeypot database, so you can then proceed to reject the retry of the spam to a legitimate email address).
- requires no per-user work, such as "training" of filters.
- requires no changes to any software, except MTAs (and only a handful of them handle most of the world's software). no new laws.
- no false positives. to get blacklisted you *must* have transmitted email to an address that could only have been obtained by illegally harvesting a website.
- even compromised home systems are not terribly harmed. if a spammer takes over your home computer and uses it, well, the IP blacklist need not be permanent, just long enough to cover a single spam run -- a few days is probably plenty. if the spammer is blasting out runs from your home computer continously, well then you have worse problems than finding yourself unable to send email to GrandMa.
- not easy to defeat. right now, anti-spammers must work very hard to locate the "real" email amidst all that spam -- and never, ever mistakenly reject a "real" email. greylisting plus honeypot RBL inverts the equation. the spammer must make sure that not a single "bogus" email address is anywhere in his database! spammers are ingenious, but developing absolutely perfect lists of legitimate email databases is something they have no experience with so far.
- no restriction of free speech. total whacko strangers who aren't spammers can still send you email -- it may just get delayed for an hour or so (a fact which is totally true already).
- nobody makes any money off it. you don't have to pay anybody, except for the effort involved in setup and maintenance (a fraction of the total time wastes on spam currently).
- computationally cheap. most MTAs are already looking up IP addresses and target addresses in databases. cost of this scheme should not greatly slow down most MTAs. especially compared to content-examination schemes such as Bayesian filters.
- no judgement calls in blacklisting. no third party has to decide what is spam and what is not. the rbl in this scheme is totally generated from absolutely bogus email addresses -- the only way you can get in the rbl is to flat-out declare yourself a scumbag by sending to one of those illegally obtained addresses.
No scheme is perfect, but greylisting combined with an RBL that is derived solely from bogus email addresses is pretty damn good.I have a partial solution that hits one item on the list ("Extreme stupidity on the part of people who do business with spammers"), but I still think it's worth a try. It's called "Spammers are Scammers." We create a TV/radio/print/web advertising campaign to drive home the point that all spammers are scammers, selling fake products, stealing credit card numbers, lying about taking you off their lists, etc. Anyone who buys anything from them is humorously but mercilessly mocked as an idiot. The ads would be created cheaply with volunteer labor and contributions, and run as free public service spots. The goal is to make it common knowledge that buying from spammers is stupid, the same way Smokey the Bear taught generations about preventing forest fires.
Yes, I know this isn't a 100% solution. However, it requires no new laws, technology, taxes, blacklists, whitelists, or anything else. It's 100% voluntary and could be run in an Open Source way. Yes, it smears all spammers with the same brush, but is any spammer going to step forward to sue? I doubt it. If it only convinced one spam-responder in five to not respond, it would be a huge hit on the spam industry.
Q: What does the "B." in Benoit B. Mandelbrot stand for? A: Benoit B. Mandelbrot
It did? Apple's Mail.app uses a Bayesian filter, right? Salting messages with random words haven't thwarted its filter at all. I might see a couple or three spam every week, but considering that's out of hundreds filtered per week with no false positives, I can live with that.
He also makes the following curious claim:
Is this really a problem? I'd say this is one of Bayesian filtering's advantages.
So far, Bayesian filtering has worked wonderfully for me. I don't see that it's been defeated -- or will ever likely be truly defeated -- at all.
Okay folks... move along... nothing to see here...
Does the author really think that I'm going to exchange formulae with everyone I want to exchange e-mail with? Even if the client software made it as easy as "pairing" bluetooth devices... ugh!
Every time I see one of these doomed-to-fail spam stopping schemes, I become more and more convinced that the only way that this problem is ever going to get solved, permanently, is with certificate-signed e-mail. Basically, e-mail client software would cryptographically sign each sender's outgoing mail and the receiver's software could check that their cert was signed by a trusted certificate authority. Most software can already do this; all you need to do is go get a certificate.
Ultimately, it would probably be left up to the individual receiver as to which certificate authorities they wanted to trust (ie, PGP's "web of trust"). But, for the most part, I think most people would default to trusting a handful of "big" cert authorities. On the face of it, there is some loss of privacy, but the loss of privacy would be in proportion to the clout of the CA that signed your certificate.... which, in turn, would be in proportion to how reliably you wanted your e-mail to be delivered. So, the sender would still get to pick how much privacy they sacrificed.
But I just see no other way to stop spam than this. Certificates would add a high degree of confidence that the sender could be reached (either by the receiver or by law enforcement)... and "reachability" is the first step towards accountability. Now, for the cases where someone managed to get an certificate with bogus contact info... well, that's what certificate-revocation lists are for. Basically, it's not really different from the IP blacklists that we're using now, except it would (hopefully) be a lot harder to obtain a new certificate than it is to obtain a new IP.
Here's what I use:
Sneak Email
Don't fear spam from shopping online ever again.
The original disposable email service. Regain power over your inbox from commercial forces, and catch them spamming.
Fully user supported and operating free of exploitable commercial ties. No debt, no operating loss, fully self sustaining... a virtual vault for your email address.
Now with version 2.0 free and premium services.
Quick start: three easy steps to total spam control.
1. Create an account: Providing a username, a password, and an email address you wish hidden from spammers.
2. Every time you need to give out your email address to somebody you don't trust, log in to Sneakemail and create a new Sneakemail address.
3. Give this Sneakemail address to them instead.
Mail sent to this Sneakemail address is rerouted to your real address, and when you reply it is rerouted back to the sender. Your real address is never seen. If you receive unwanted mail through this Sneakemail address, such as spam, you can take control by either filtering incoming mail using the Sneakemail filters, disabling the Sneakemail address itself, or disposing of it permanently. You also now know where a spammer got your address.
You now know all you need to know to protect your inbox from the internet by using Sneakemail.