Slashdot Mirror
Bill Gates Forecasts Victory Over Spam
nfk writes "BBC reports from the World Economic Forum at Davos, where Bill Gates said spam will be a thing of the past in two years' time, thanks to a three-pronged approach to the problem: filters, expensive computation for e-mail and the digital equivalent to stamps, paid if the receiver considers he is being spammed. He also expects to catch up with Google, although he praises the company and the IQ of its research team. Finally, he announces mind blowing developments for the next XBox generation and says that, in a decade from now, 'we will laugh at personal computing as we know it.' No need to wait, I do it every day." (We've mentioned Microsoft's sender's-option payment scheme before.)
KFilter and GFilter, cheap OSS knock-offs of whatever Bill implements to combat spam, repelete with a /. summary with an editors savage addendum bashing Bill Gates, the main inspiration for software for OSS hippies to rip off.
Asked whether Microsoft missed the boat in the field of search technology, Mr Gates admitted that he had to take the blame for losing out to Google.
"We took an approach that I now realise was wrong," he said.
I may not like Bill Gates and the way his company acts, but I have to give credit to a man who can admit his mistakes. It's not an easy thing to do.
--
In London? Need a Physics Tutor?
American Weblog in London
You may have some latency issues on your connection to be worked out.
...the digital equivalent to stamps, paid if the receiver considers he is being spammed.
As much as Bill Gates and Microsoft get group-hated there are some good ideas and some possibilities for decent implementation here, such as this. It is the darker side of MS that holds them back; if they could make great software that was fully transparent (I'm sure most of the developers would be happy with this) they would be totally win-win, and Bill Gates seems pretty philantropic as an individual, I wonder what holds them back...
MS is not an average company in the pocket of suits, it is run by an intelligent guy (by far the best programmer, but a very intelligent all-rounder) who has some kind of vision. I see, not too far from now, a bright future with Gates and Torvalds hand-in-hand. [No, my name is not Morpheus].
--
FreeNET user? Comfortable with the adverse selection?
If microsoft managed to find a way to make money off of spammers then "geeks" who don't currently spam now, may start doing so just to mess with them.
Sort of like trying to thwart the microsoft security initiative.
I am not saying it is right, but that it would happen.
However, spam is a problem. It is almost impossible to have a "permanent" address anymore and that sucks.
I would like to hear about solutions that don't involve paying microsoft anything.
--ken
--ken
Bitcoin pyramid: Join here: http://www.bitcoinpyramid.com/r/1427 it's FREE!
I am more intrested in an approach that can rank the level of attention that I should pay to e-mail. I'd like to have a white list that allows me to set different priority levels based upon the sender. I'd like to give a higher priority to mail that has a valid signature. I'd also give a higher priority to mail from people in my address list.
By the way, which e-mail clients meet your criterion for a "real mail clients"?
I am still trying to figure out where I can purchase the Monty Python E-Mail Client.
"force the sender of an e-mail to pay up when an e-mail was rejected as spam"
That would be a good idea for phone calls from people trying to sell you stuff.
There's lots of great filtering technologies available out there, and the best ones are non-commercial in nature. Microsoft or Yahoo have not helped my spam situation; but spamprobe, bogofilter, spamassassin, and spambayes definitely have helped me, in very real terms: > 99% accuracy, with (generally) zero false positives depending on the quality of configuration.
Now an appeal to you folks out there who use these filters I've mentioned with similar good results (w.r.t. accuracy): we no longer see spam thanks to our filters. How about taking it one step further? Join the WPBL project and help us centrally collect IP addresses of spammers. It's an automated system to determine real-time spam sources using reliable, trusted data contributors. We are currently tracking over 15,000 IPs.
Damn straight. I use Mail.app on my Macs. After a few weeks of training, these days I essentially receive no spam. About one message every two weeks will get through. Usually when that happens it reminds me to empty the 700 spam messages out of my junk folder. A quick scan assures me that, once again, no false positives.
For Mac users, spam is already a thing of the past.
We don't have a state-run media we have a media-run state.
That's an interesting comment, but at the risk of getting modded down, I have to ask:
In what ways do Bill and/or Microsoft impede yours (or anyone's) ability to improve software?
I'm not trolling here, I'm seriously cusious. Thanks in advance for your reply.
everything in moderation
I realize you're trying to be funny, but the sad reality is that whatever the solution is(if there is one), it will only work if there's enough mail clients and servers that apply it, and the matter of fact is that Microsoft holds the keys to a very large client base. While they alone can't do it, they must certainly be part of the solution for it to work. So while we may despise Microsoft, the fact is on this issue they both are on our side, and we WANT them here.
I applaud any effort that will reduce spam and send the spammers to jail. Perhaps some day, we can have spam-free email again like in the good old days...
If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...
Does this mean you would need to provide a valid credit card number to set up an email account? That's done already if you go through an ISP, but what about all those free, web-based email servers? Or what about people who have set up their own email server on a PC? How would you go about tracking down these people and billing them?
There is one thing we have all learned from the spammers and that is that they are smart. They have just as many smart programmers working for them as we have fighting against them. They know how to avoid detection. Spam and identity theft go hand and hand. So if they were financially responsible, whose to say they wouldn't just fork over a stolen credit card number and have Joe Sixpack pick up the tab?
"Oh dear, she's stuck in an infinite loop and he's an idiot" -Prof. Farnsworth (Futurama)
SpamAssassin uses a scoring system to determine the "spamminess" of a piece of mail. Each test in SA has a score assigned to it by some fancy GA algorithm. The way I do it is sort my incoming mail by the SA score and pay attention accordingly.
Micropayments don't have to change SMTP at all. The client can discard or bounce the message if it doesn't have appropriate payment. In fact, this is probably the better way to do things since it puts control of what to receive in the hands of the recipient, not the sender or some mail server (which is what caused the spam problem in the first place).
-- Ed Avis ed@membled.com
Until it is illegal to send someone email i cant really fathom how you could stop spam? If sending email becomes hard or expensive some bozo will reinvent email and people will flock there instead.
A ban against email while regular IRL spam is allowed is also pretty inconsistent. Maybe if we put some pressure on the companies SENDING the spam we could get some results. Just plain boycott any company that sends spam and the problem will stop pretty fast. Why not start a list with the worst offenders (companies, not the spammers).
Without companies giving the spammers money the problem wouldnt exist.
Cure the illness not the symptoms!
HTTP/1.1 400
MS has 95% of users hooked on an ancient browser, which means my web-based applications must continue to use old old techniques.
"I realize you're trying to be funny, but the sad reality is that whatever the solution is(if there is one), it will only work if there's enough mail clients and servers that apply it, and the matter of fact is that Microsoft holds the keys to a very large client base."
That sounds like a false premiss.
Current Baysian (sp?) filtering works just fine without a lot of users. In fact, now that so many mail programs are using this technique the spammers have adapted to it by including words in their messages to get through the filtering.
Furthermore, they are including large lists of words which will eventually cause your filtering mechanism to filter out legitimate mail. By the time MS has its filtering system ready the entire concept will have been used up IMHO.
I've had good luck with Mailblocks.com. No training needed. The only way spam gets through is if the spammer takes the time to visit a web page, squint at a graphic and type in a word. The few small time spammers that have done this in my case have then been explicitly blocked.
I predict MS will scrap all their anti spam work and start over before 2006. Maybe they will come up with something good. But everything being said by Bill Gates at this point is just marketing hype, not valid design concepts (for which he is not qualified).
That would kill the problem at its source.
More seriously, you could probably remove a good portion of the spam short of this draconian step, but it would probably require:
1. Verification of the return address given in e-mail.
2. E-mail being held on the originator's side until requested by the recipient.
For example, you send an e-mail. The recipient's server then sends a one-time key back to the return address on the email. The originating server then includes this key and a link to the body of the message in the e-mail header and sends it back. The link sits on the recipient's mail server until that person either reads the message by clicking on the link to download it, or deletes the link thus removing the key.
The nice thing about leaving the message on the originating server is that spammers would have to give valid return information, and they have to store the spam on their server until someone requests it. There would be higher up-front bandwidth and data storage costs from the verification process, especially for the more prolific spammers, but it would probably lower the overall bandwidth required since header info is usually much smaller than the message itself, and deleting it prevents the larger message from being transmitted. It would also probably slow the spread of many e-mail viruses (and make them far easier to track), because a really prolific one would fill up the originating server with a bunch of garbage while waiting for a response, and they wouldn't be able to mask the return address by giving a phony one.
This tagline is copyrighted material. Please send $10 for an affordable replacement.
My idea for reducing spam by at least getting rid of a whole load of joe-jobbing would be to let people announce how to verify emails from them (I've received something like 50,000 bounces as a result of some spammer sending mails from hijacked machines claiming to be from [random-word]@schmerg.com).
I own all email sent from schmerg.com, so I add a (new type of) DNS record of my public key, and then every email that I send I add a header "X-WonderSchemeEncyrptedChecksum" with the value of the SHA-1 checksum of that message's body as sent, encrypted with my corresponding private key.
If your mail system doesn't know about this, nothing changes, but if you DO know about the scheme, then whenever you receive an email you do a DNS lookup on the sender's domain. If that domain has no key listed, then you're none the wiser, but if they DO have a key listed (and here my domain schmerg.com does) then you can safely reject any emails that don't have the new header, or where decrypting the checksum fails to match the body.
This way an organisation can still add their crappy sigs or whatever, and then sign all their email, and spammers will learn not to use that domain in their From address.
Big ISPs and people like HotMail can sign all the email their users send thru their system, and we start to reduce the ability of spammers to have false From addresses. If you want to send email claiming to be from a domain protecting itself in this way, you have to send it thru that domain at some point (or know the private key yourself).
It's nowhere near a complete solution to spam, but it makes life harder for spammers (and phishers and the rest), and it rewards those willing to make the effort without punishing those who don't.
To get round various implementation issues you'd probably want to add multiple keys to your DNS record and then describe which one you were using for each email (so you can rotate keys, or use different keys for different locations, and phase out old keys regularly if you're Hotmail.com or similar), but DNS propagation, caching and lookup is a given on today's internet.
If you can't be bothered checking the identity of the sender you don't have to, but if you want to (and you can afford the DNS lookup and the cycles to checksum the message etc.), then you can.
--
Tim
I spent a lot of money on booze, birds and fast cars. The rest I just squandered. - George Best
For a micropayment, the cost to a single mistake would be small enough that you wouldn't care. It costs me about 30 cents to mail a letter, if once in a while I had to pay 2 cents because someone mistook my email, I can afford it. A spammer cannot however afford all the recipents of his spam charging 2 cents because it adds up
Unfortunatly I don't know if it is worth the effort to hit the charge sender button. Means I have to sign up for a lot of things, for little appearent gain.
The bigger problem with this though is real mailing lists. Its easy enough to sign up for the countrpane newsletter on a lot of accounts (script), and then (again scripted) when a newsletter arrives hit the charge button.
And I don't think micropayments will stop spam - wouldn't the spammers just use servers that didn't require that?
It's your server at mailinator.that counts. It can refuse to accept email except from people (or other mail servers) who pay.
And would email be as useful if you could only get mail from someone who bought into a particular micropayment system?
The payments Microsoft is proposing aren't necessarily monetary. Sometimes it can be a hard computational problem, which takes you a few seconds to compute. Spam depends on the very low cost of email. If you have to buy 10 computers to send your spam, instead of just one, it's suddenly far less profitable. Whereas you yourself can easily afford a few seconds added to each of the few dozen emails you send each day, since almost every personal computer has free cycles.
Of course, that depends on spammers to use their own computers. If they're using yours, a problem which plagues Microsoft-based computers, you're still stuck.
I don't think that would bother most people. By "most people" I don't of course mean "most slashdotters." I mean all those who are already locked into Windows and don't mind, to whom the vast majority of spam is directed, and which most likely contains all the people who are actually dumb enough to respond to spam. Make spam infeasible for that group of people, and you make spam infeasible full stop.
This type of idea is going to put small guys out of business. I run a non-profit online service (auctions) that doesn't compete on the level of eBay - but is a competitor none the less. If I had to absorb a heavy computational expense for every one of my two or three thousand daily system emails (auction notifications, registrations, etc) and/or an actual per-message "postage fee", I could not compete at all. Big players like multi-billion-dollar eBay and such would do well, but small guys like me who run non-profit, free sites would be shoved out of the way.
As it is, I'm already pissed that AOL is classifying auction notices and registration confirmations from my site as "spam". I get about half a dozen emails every day from AOL users complaining that "your site never sent me a registration confirmation" or "I'm not getting auction-closed notices" and it's because AOL is deleting them or dumping them into the users spam folder, which most users never bother to check either.
I envisage that the amount of computation could be variable by the client, and it would be one of several factors weighed. For example with Spamassassin you might see something like
HTML.........1.0 points.....Message contains HTML
HASH_CASH....-3.5 points....Hash cash payment of 35 computrons
Total score: -2.5 points ==> not spam
As usual, the Spamassassin developers would look at their corpus of spam and ham and derive the right weighting for different amounts of hash cash postage. Users could tweak it themselves if they wanted.
-- Ed Avis ed@membled.com
The breakage problem has nothing to do with "a few non standard systems" NT updates were notorious for breaking popular non MS apps. It was bad enough that windows admins became afraid of patching their machines thinking the kiddy potential was the smaller risk.
Even XP SP 1 was known to prevent some of our office systems from booting.
The problem is alack of Q&A.
problem is that the number of bits of collision found is a probabilistic event. You always have at least the number you requested but sometimes you can have as much as 10 or 15 bits more because that is just what you stumbled across in search for the collision. It's always safest to say whether or not it passed the minimum number of bits collision threshold and not that it has a certain number of bits collision.
I suggest you try this using the hashcash executable. Run the process for about a week and log the number of collision bits found versus number of times it was found. Its quite illuminating.
Hmm, so the 'amount' of hash cash postage is probabilistic, but then so is the determination of what is spam and what isn't. It is unlikely that a spammer would run the hash cash code and get very good luck to hit long collisions by accident, so the length of collision found is a reasonable indicator of the computing time put in.
Correct me if I'm wrong - but surely a collision of 6 bits could not take any less time to find than one of 5 bits, and quite likely would take longer. So, a longer collision should be treated as better, though the probabilistic weighting you give to this might have to be carefully chosen.
-- Ed Avis ed@membled.com
obviously, we need to have a longer conversation. Feel free to contact me directly via the link on the contact section of the camram web site.
This is true of all proof of work systems. You could get really lucky and meet the criteria for "done" on the first try. On average however you will take the target amount of time. Which means sometimes it will take longer and sometimes it will take shorter to reach "done".
Now on average, every time you increase the cost of a stamp by a bit, you double the average cost. So if a 22 bit stamp takes 15 seconds on average, a 23 bit stamp will take 30 seconds on average. Now it's also possible to encounter a 26 or 32 bit stamp in the search for a lower value one. There's no magic or exploitation involved, it's just how sha1 and the search for the right completion work. Think dumb-F'n-luck. which is why I choose to use the desired number as a predicate and use a simple go/no go. Other interpretations are possible but less predictable.
Seriously, contact me directly and I highly recommend playing with the hashcash code from hashcash.org and really get a good feel for what it means to generate stamps. There's nothing like hands-on experience at this point.
---eric
I personally use KMail and POPFile. I hear Thunderbird is good, and its integrated spamfilter is cool. And I'm sure Emacs would suffice. My one gripe about KMail and POPFile is that they aren't well enough integrated. If mail gets misclassified, just dragging it to the right filter ought to train the spamfilter too. A POP proxy and web interface is cool, but there ought to be a command line interface for spam filters that mail clients could automatically invoke.
Spam filters, whitelists, computation, and even micropayments, as ways to prioritize mail, each have their costs. All can result in important messages being lost. Computation and micropayments both make it harder to communicate, which I don't think is a good idea. I think the best long-term solution is to make it impossible to hide where mail is coming from. Then, legislation against spam will be effective, and in countries without such legislation, overseas bandwidth providers can pressure ISPs to drop their spammers. Combined with better security to stop zombies and filters to catch thre rest, spam can be eradicated.
Litigious bastards
The biggest problem that they all break the simple model that makes email work. Users can pass an "email address" by any means (inband or out of band) they want, and then they can exchange messages. Any kind of payment system will require a security relationship between the email-exchanging parties. Security realationships are expensive, and tend to scale as O(N^2).
Increasing the cost (CPU or money) would still let "rich" spammers spam, but would shut down mailing lists, and make a big extra barrier for people to freely email each other. (And no, whitelisting the mailing lists won't work -- because the spammers would just forge mail from those mailing lists.) Getting rid of the "poor" spammers would be nice (no more herbal viagra...) but would encorage big companies to spam (and they would claim that this is legitimate.) Consider this, as well: much spam these days is delivered by zombies -- is it really costing the spammer anything if his network of zombies has to do a little more CPU intensive work?
If you require a micropayment with each email, that means you either have an extra step to take with each email (insert smartcard, type pin, or whatever) or your MUA does that for you. The previous is enough difficulty to kick many non-technical users off the 'net. The later would imply that malmalware or a social engineer can steal all your email money.
There are lots of ways to help reduce spam (currenly more than 50% of email is spam.) Filters help a lot, and the ASRG is working on new barriers to spammers. If CAN-SPAM were enforced, it would make a large dent in the amount of spam (and make the rest easier to filter.) I think that has to be the magic bullet for spam, if there is going to be one. Filters and other barriers may slow spammers down, but if there is no penalty for trying, they'll keep coming until they find a way to circumvent the filters, the payment schemes, etc. The magic bullet canot be filtering alone -- I'm pretty sure that well-written spam would require a turing test to distinguish from ordinary email.
Did Bill means his team is going to *invent* Baysian spam filtering? I am used to this in Mozilla for a long time.
http://www.ieaa.org/~adrian/
The problem with your idea, and Yahoo's Domainkeys, are as follows:
I think a far better better proposal for what you want to do is Sender Permitted From (SPF). It has been mentioned quite a few times on /. and elsewhere.
SPF support for most open source mail servers can be found at libspf2.
Is that every one of Bill's solutions have been done FIRST in the Open Source community. The BBC mentioned two concepts that I remember:
1: Filters (Since when does Outlook or OE have Bayesian filtering capabilities?)
2: Causing spammers to pay a certain price. This is also being done for example, by requiring every subsequent attempt to send an email to a non-existant address forceing a cumulative delay in responding to the next attempt from the same host (this has been discussed on the Qmail lists quite a bit).
MS EXchange, IIRC, doesn't even check to see if there is an MX record for the originating domain! Sendmail even does that. How many hotmail messages do we get from xdtty@weftre.wdt (obviously nonexistant domains). Obviously Hotmail doesn't check either (when I pointed this out to them, I also pointed out that Sendmail DOES check these things)
Bill should mean "We want to be the first proprietary vendor to copy the methods of the Open Source solutions to the Spam Problem." It would have been more accurate.
Note that the above solutions are SMTP compatible and require no protocol extensions. They would have the effect of rendering SPAM less effective, and harvesting email addresses more costly.
LedgerSMB: Open source Accounting/ERP
Yes, but there's nothing to require the spammer to wait for the first connection to finish before starting another one.
Couldn't a spammer get around that simply by using a multithreaded process to send the spam? At any given time most threads would be idle waiting for an SMTP connection, but they wouldn't be using any CPU time. The spammer might have to do some tuning to find the right number of threads to use, but it seems to me that properly tuned, the overall throughput would be the same as it is now.
--Stuart