Slashdot Mirror

Frequent Slashdot contributor Bennett Haselton writes "An estimated 200,000 Hotmail users currently have their auto-reply set to a message spamming an advertisement for Chinese scam websites, which sell "discounted" electronics. Presumably the spammers compromised a large number of Hotmail accounts to pull this off, but wouldn't it be pretty easy for Hotmail to query for which users have that set as their auto-reply, and turn the auto-reply off for them?" Read below for Bennett's thoughts.

After a recent mailing that I sent out to a subset of my proxy mailing list, I got back 18 auto-replies from Hotmail users, all substantially similar to this:

Dear friend:
We are an electronic products wholesale .Our products are of high quality and low price. If you want to do business , we can offer you the most reasonable discount to make you get more profits. We are expecting for your business.

Please visit our website: www.wedosale.com

Email: wedosale@vip.188.com .
MSN: wedosale@hotmail.com .

Looking forward to your contact and long cooperation with us!

Our mainly products such the phones, PSP, display TV, notebook, video, computers, Mp4, GPS, xbox 360, digital cameras and so on.

Welcome to visit our website!

Some of the spam auto-replies advertised different websites, and the wording varied between the different auto-responses, but they were all similar advertisements for Chinese electronics "retailers." (And so, I assume, the websites are all fronts for the same company -- if multiple spammers had independently hacked Hotmail users' accounts to set their auto-replies, it would be vanishingly unlikely that those spammers would all happen to be electronics hawkers.) This was from a mailing that I sent to a set of subscribers that included about 26,000 users with "hotmail.com" e-mail addresses. If 18 out of 26,000 users in my sample have had their accounts hacked to send spam auto-replies, then this must be happening to a large number of Hotmail users -- not a large proportion (only one in 1,500, in my sample), but with about 300 million Hotmail users, that would still be a large absolute number.

The same spammers have apparently been spamming through Hotmail auto-replies for at least 11 months, according to this post in the Windows Live Help community forum from January 2009. At first, some pundits seemed to have assumed that spammers had created these accounts themselves and subscribed the accounts to people's lists, in order to spam the list owners (and, if it's a list that accepts subscriber posts, broadcast the spam to the other list readers). However, looking at the addresses in my proxy mailing list that were sending the spam auto-replies, I noticed that (1) our records show that the auto-reply-spamming subscribers joined the mailing list by various means, signing up through different Circumventor websites, not indicative of how a spammer would have joined the list by automated means, and (2) many of their email addresses are associated with legitimate-looking Myspace and Facebook accounts. Thus it looks as if these were real users who joined the list legitimately, and then got their accounts hacked by the spammers, who set those users' accounts to send the spam as an auto-response.

(If you happened to look at the spammers' www.wedosale.com website, at this point you might be thinking: I don't want to give money to spammers, but can I really get a Blackberry for only $295? Couldn't I just order from the website, and then if the goods don't show up or they're not as advertised, I can dispute the charge on my credit card? Well, I signed up for a dummy account on the www.wedosale.com page and got as far as the order page, and the only payment types that they accept are wire transfer, Western Union, and Moneygram -- precisely those types where you cannot get the money back or dispute fraudulent charges. If you've already gone and ordered a Blackberry, don't hold your breath.)

If my 26,000 users were a representative sample of the 300 million current Hotmail users, then with 1 out of 1,500 users in my sample being "infected," I could estimate that about 200,000 Hotmail users (1/1500 times 300 million) are currently set to send spam auto-replies. Hotmail claims to process 3 billion non-spam e-mails per day, for an average of about 10 non-spam e-mails per Hotmail user. That's the average for all users; what's the average for the infected users? Some factors would tend to lead to a lower average for infected users -- if they have lots of friends sending them mail, it's more likely that one of their friends would have told them about the auto-reply spam and told them to turn it off, so perhaps the users still sending the spams are the ones who don't receive a lot of messages from their friends. On the other hand, some of the infected accounts may be receiving more (non-spam) e-mail than average; one reason people sometimes abandon webmail accounts is that they're getting too much mail, even from newsletters like the Circumventor list that they had legitimately subscribed to. So, figuring that factors in both directions roughly cancel out, if each infected user is receiving the average number of 10 emails per day and sending 10 auto-reply spams in response, that's still a total of 2 million outgoing spams per day shilling for nonexistent Chinese iPhones.

These are just back-of-the-envelope calculations, but even I'm overestimating by a whole order of magnitude, that's still 0.2 million auto-reply spams per day, or about 70 million spams that will be sent by this one company through Hotmail's servers in the coming year, if Hotmail doesn't stop it. (And closer to a billion spams in the coming year if I'm not overestimating.)

And it's actually worse than that, because these spams are less likely than average to be filtered, since they're coming from Hotmail's servers. Normally you'd think that the content-based module of a spam filter would have no problem catching a message like the one at the top of this article, especially if millions of similar messages have been spewed out over the past year. However, messages from Hotmail's servers, regardless of content, are less likely to be blocked, since their network has a good reputation for sending little spam overall (due to measures such as requiring users to fill out a CAPTCHA when signing up, blocking each account from sending more than 500 messages per day, etc.). When I sent messages to the infected Hotmail users from my Gmail account, to see if the auto-responses would get through Gmail's spam filter, Gmail's blocked only half of the replies. When I mailed all the users again from my Hotmail account, the results were strange -- most of the users' accounts sent back no auto-reply at all, not even a reply that got routed to my junk folder. (Why would Hotmail accounts not send an auto-reply in response to a message from a Hotmail user? Please post if you have any idea what's going on there.) However, of the infected Hotmail accounts that did send a spam auto-reply, 100% of those auto-reply spams were delivered to my inbox. (Apparently, Hotmail's spam filter usually assumes that messages from other Hotmail users can't possibly be spam.) Only Yahoo Mail's spam filter, when I sent a test message to the infected users from my Yahoo Mail account, blocked all of the auto-replies as junk mail.

For the infected users on my mailing list, I sent them a link to a set of instructions I'd written about how to set and un-set their Hotmail auto-reply and how to change their Hotmail password, with the hopes that they'd eventually see the message and follow the steps. 18 users rescued, 200,000 to go.

So this is basically what's happening, but it still leaves some unanswered questions, such as: Why Hotmail accounts, but not Yahoo Mail, GMail, or AOL accounts? I've never noticed any auto-reply spam sent from any accounts at any of those other services. Whatever the spammers did to gain control of so many Hotmail accounts, if it was profitable for them, why didn't they do the same thing for Yahoo Mail? And, why did only one spammer do this? If they're sending between 1 and 10 million spams per day for free, they're probably making money at it. Whatever they did to hack those accounts, why wouldn't other spammers figure out the same method and copy them?

Presumably the Chinese spammers stole large numbers of passwords from Hotmail users either via a huge phishing attack, or through a security hole in Hotmail or some other part of the Windows Live service. If it was done via a security hole in Hotmail that the spammers discovered, then that would explain why the spammer's methods only worked for Hotmail accounts, and also why no other spammers have copied their techniques. (A phishing attack, on the other hand, would be easy to modify for other webmail services, and would also be easy for other spammers to emulate, so that's not consistent with the observed evidence so far.) I also found this post from blogger Stuart Shelton describing how his account was hacked by Chinese spammers -- and from the blog post, it's clear that he's very tech-savvy and would have been unlikely to fall for a run-of-the-mill password phish. If the attack happened even to people who know what they're doing, that seems to make the security hole explanation more likely.

Perhaps others can come up with some theories about what happened. It's easy to come up with guesses, but the hard part is to reconcile them with the fact that it has only affected Hotmail users so far, and no other spammer seems to have figured out how to copy the same technique yet.

But there's a much simpler question too: Why doesn't Microsoft just turn off the auto-replies for these users' accounts? They can query to see exactly which users have these messages in their auto-replies, and then un-set the auto-reply automatically. Yes, I know that even for a simple database operation like that, there's always more to it when you're managing hundreds of millions of accounts across multiple servers -- but if it will stop this one sender from sending between 50 million and 500 million spams (that in many cases will bypass people's spam filters) from Hotmail's servers in the coming year, isn't it probably worth it?

And even if it wasn't a phishing attack this time, sooner or later some other spammer will probably capture tens or hundreds of thousands of Hotmail accounts using a phish or some other method, and try spamming through auto-replies as well. So if Hotmail "fixes" this batch of auto-reply spam for practice, then the next time it happens, they'll know exactly what to do to take care of it.

I've written some columns where I strongly believed every word but expected a lot of opposition, some where I wasn't sure if I was right and just wanted to see what people thought, and . But I rarely argue something that I think is a no-brainer. Hotmail should un-set the auto-replies for those users whose accounts are spamming for nonexistent Chinese electronics knockoffs, before those accounts send another several hundred million spams in the coming year. Am I smoking crack?

Then again, maybe expectations for Hotmail shouldn't be set too high. I use SpeakEasy for my mail provider, and on about November 19th I found that all messages sent to hotmail.com addresses from SpeakEasy's servers were being bounced with an error message rejecting them for "spam-like characteristics."I called SpeakEasy and they confirmed that they knew Hotmail was blocking all mail from their users (although for "security reasons," SpeakEasy couldn't tell me what they were trying to do about it). The block wasn't lifted until about November 28th, when my messages started getting through again.

If SpeakEasy, which has been in business for 15 years, has annual revenues of $60 million, and was bought in 2007 by Best Buy, can't even get through to Microsoft in less than 10 days to tell them to stop blocking all mail from their servers, then Microsoft should first fix their postmaster trouble ticket system, so that people are not blocked from writing to their friends and family members at Hotmail for a week and a half. Then get to work on the spam auto-responders.

Frequent Slashdot contributor Bennett Haselton writes "An estimated 200,000 Hotmail users currently have their auto-reply set to a message spamming an advertisement for Chinese scam websites, which sell "discounted" electronics. Presumably the spammers compromised a large number of Hotmail accounts to pull this off, but wouldn't it be pretty easy for Hotmail to query for which users have that set as their auto-reply, and turn the auto-reply off for them?" Read below for Bennett's thoughts.

After a recent mailing that I sent out to a subset of my proxy mailing list, I got back 18 auto-replies from Hotmail users, all substantially similar to this:

Dear friend:
We are an electronic products wholesale .Our products are of high quality and low price. If you want to do business , we can offer you the most reasonable discount to make you get more profits. We are expecting for your business.

Please visit our website: www.wedosale.com

Email: wedosale@vip.188.com .
MSN: wedosale@hotmail.com .

Looking forward to your contact and long cooperation with us!

Our mainly products such the phones, PSP, display TV, notebook, video, computers, Mp4, GPS, xbox 360, digital cameras and so on.

Welcome to visit our website!

Some of the spam auto-replies advertised different websites, and the wording varied between the different auto-responses, but they were all similar advertisements for Chinese electronics "retailers." (And so, I assume, the websites are all fronts for the same company -- if multiple spammers had independently hacked Hotmail users' accounts to set their auto-replies, it would be vanishingly unlikely that those spammers would all happen to be electronics hawkers.) This was from a mailing that I sent to a set of subscribers that included about 26,000 users with "hotmail.com" e-mail addresses. If 18 out of 26,000 users in my sample have had their accounts hacked to send spam auto-replies, then this must be happening to a large number of Hotmail users -- not a large proportion (only one in 1,500, in my sample), but with about 300 million Hotmail users, that would still be a large absolute number.

The same spammers have apparently been spamming through Hotmail auto-replies for at least 11 months, according to this post in the Windows Live Help community forum from January 2009. At first, some pundits seemed to have assumed that spammers had created these accounts themselves and subscribed the accounts to people's lists, in order to spam the list owners (and, if it's a list that accepts subscriber posts, broadcast the spam to the other list readers). However, looking at the addresses in my proxy mailing list that were sending the spam auto-replies, I noticed that (1) our records show that the auto-reply-spamming subscribers joined the mailing list by various means, signing up through different Circumventor websites, not indicative of how a spammer would have joined the list by automated means, and (2) many of their email addresses are associated with legitimate-looking Myspace and Facebook accounts. Thus it looks as if these were real users who joined the list legitimately, and then got their accounts hacked by the spammers, who set those users' accounts to send the spam as an auto-response.

(If you happened to look at the spammers' www.wedosale.com website, at this point you might be thinking: I don't want to give money to spammers, but can I really get a Blackberry for only $295? Couldn't I just order from the website, and then if the goods don't show up or they're not as advertised, I can dispute the charge on my credit card? Well, I signed up for a dummy account on the www.wedosale.com page and got as far as the order page, and the only payment types that they accept are wire transfer, Western Union, and Moneygram -- precisely those types where you cannot get the money back or dispute fraudulent charges. If you've already gone and ordered a Blackberry, don't hold your breath.)

If my 26,000 users were a representative sample of the 300 million current Hotmail users, then with 1 out of 1,500 users in my sample being "infected," I could estimate that about 200,000 Hotmail users (1/1500 times 300 million) are currently set to send spam auto-replies. Hotmail claims to process 3 billion non-spam e-mails per day, for an average of about 10 non-spam e-mails per Hotmail user. That's the average for all users; what's the average for the infected users? Some factors would tend to lead to a lower average for infected users -- if they have lots of friends sending them mail, it's more likely that one of their friends would have told them about the auto-reply spam and told them to turn it off, so perhaps the users still sending the spams are the ones who don't receive a lot of messages from their friends. On the other hand, some of the infected accounts may be receiving more (non-spam) e-mail than average; one reason people sometimes abandon webmail accounts is that they're getting too much mail, even from newsletters like the Circumventor list that they had legitimately subscribed to. So, figuring that factors in both directions roughly cancel out, if each infected user is receiving the average number of 10 emails per day and sending 10 auto-reply spams in response, that's still a total of 2 million outgoing spams per day shilling for nonexistent Chinese iPhones.

These are just back-of-the-envelope calculations, but even I'm overestimating by a whole order of magnitude, that's still 0.2 million auto-reply spams per day, or about 70 million spams that will be sent by this one company through Hotmail's servers in the coming year, if Hotmail doesn't stop it. (And closer to a billion spams in the coming year if I'm not overestimating.)

And it's actually worse than that, because these spams are less likely than average to be filtered, since they're coming from Hotmail's servers. Normally you'd think that the content-based module of a spam filter would have no problem catching a message like the one at the top of this article, especially if millions of similar messages have been spewed out over the past year. However, messages from Hotmail's servers, regardless of content, are less likely to be blocked, since their network has a good reputation for sending little spam overall (due to measures such as requiring users to fill out a CAPTCHA when signing up, blocking each account from sending more than 500 messages per day, etc.). When I sent messages to the infected Hotmail users from my Gmail account, to see if the auto-responses would get through Gmail's spam filter, Gmail's blocked only half of the replies. When I mailed all the users again from my Hotmail account, the results were strange -- most of the users' accounts sent back no auto-reply at all, not even a reply that got routed to my junk folder. (Why would Hotmail accounts not send an auto-reply in response to a message from a Hotmail user? Please post if you have any idea what's going on there.) However, of the infected Hotmail accounts that did send a spam auto-reply, 100% of those auto-reply spams were delivered to my inbox. (Apparently, Hotmail's spam filter usually assumes that messages from other Hotmail users can't possibly be spam.) Only Yahoo Mail's spam filter, when I sent a test message to the infected users from my Yahoo Mail account, blocked all of the auto-replies as junk mail.

For the infected users on my mailing list, I sent them a link to a set of instructions I'd written about how to set and un-set their Hotmail auto-reply and how to change their Hotmail password, with the hopes that they'd eventually see the message and follow the steps. 18 users rescued, 200,000 to go.

So this is basically what's happening, but it still leaves some unanswered questions, such as: Why Hotmail accounts, but not Yahoo Mail, GMail, or AOL accounts? I've never noticed any auto-reply spam sent from any accounts at any of those other services. Whatever the spammers did to gain control of so many Hotmail accounts, if it was profitable for them, why didn't they do the same thing for Yahoo Mail? And, why did only one spammer do this? If they're sending between 1 and 10 million spams per day for free, they're probably making money at it. Whatever they did to hack those accounts, why wouldn't other spammers figure out the same method and copy them?

Presumably the Chinese spammers stole large numbers of passwords from Hotmail users either via a huge phishing attack, or through a security hole in Hotmail or some other part of the Windows Live service. If it was done via a security hole in Hotmail that the spammers discovered, then that would explain why the spammer's methods only worked for Hotmail accounts, and also why no other spammers have copied their techniques. (A phishing attack, on the other hand, would be easy to modify for other webmail services, and would also be easy for other spammers to emulate, so that's not consistent with the observed evidence so far.) I also found this post from blogger Stuart Shelton describing how his account was hacked by Chinese spammers -- and from the blog post, it's clear that he's very tech-savvy and would have been unlikely to fall for a run-of-the-mill password phish. If the attack happened even to people who know what they're doing, that seems to make the security hole explanation more likely.

Perhaps others can come up with some theories about what happened. It's easy to come up with guesses, but the hard part is to reconcile them with the fact that it has only affected Hotmail users so far, and no other spammer seems to have figured out how to copy the same technique yet.

But there's a much simpler question too: Why doesn't Microsoft just turn off the auto-replies for these users' accounts? They can query to see exactly which users have these messages in their auto-replies, and then un-set the auto-reply automatically. Yes, I know that even for a simple database operation like that, there's always more to it when you're managing hundreds of millions of accounts across multiple servers -- but if it will stop this one sender from sending between 50 million and 500 million spams (that in many cases will bypass people's spam filters) from Hotmail's servers in the coming year, isn't it probably worth it?

And even if it wasn't a phishing attack this time, sooner or later some other spammer will probably capture tens or hundreds of thousands of Hotmail accounts using a phish or some other method, and try spamming through auto-replies as well. So if Hotmail "fixes" this batch of auto-reply spam for practice, then the next time it happens, they'll know exactly what to do to take care of it.

I've written some columns where I strongly believed every word but expected a lot of opposition, some where I wasn't sure if I was right and just wanted to see what people thought, and . But I rarely argue something that I think is a no-brainer. Hotmail should un-set the auto-replies for those users whose accounts are spamming for nonexistent Chinese electronics knockoffs, before those accounts send another several hundred million spams in the coming year. Am I smoking crack?

Then again, maybe expectations for Hotmail shouldn't be set too high. I use SpeakEasy for my mail provider, and on about November 19th I found that all messages sent to hotmail.com addresses from SpeakEasy's servers were being bounced with an error message rejecting them for "spam-like characteristics."I called SpeakEasy and they confirmed that they knew Hotmail was blocking all mail from their users (although for "security reasons," SpeakEasy couldn't tell me what they were trying to do about it). The block wasn't lifted until about November 28th, when my messages started getting through again.

If SpeakEasy, which has been in business for 15 years, has annual revenues of $60 million, and was bought in 2007 by Best Buy, can't even get through to Microsoft in less than 10 days to tell them to stop blocking all mail from their servers, then Microsoft should first fix their postmaster trouble ticket system, so that people are not blocked from writing to their friends and family members at Hotmail for a week and a half. Then get to work on the spam auto-responders.

Frequent Slashdot contributor Bennett Haselton sends in his latest, which begins like this and continues behind the link. "When Google's VP of Engineering announced their proposed Knol project, where users can submit articles on different subjects and share in the AdSense revenue from the article pages, he didn't mention "Wikipedia," but practically everyone else did who blogged about it. Here's what I think will happen, if Knol is implemented according to the plan: Even though it won't technically be a "Wikipedia fork," it will quickly become equivalent to one, with a "gold rush" of users copying content from Wikipedia to Knol articles hoping for a piece of the AdSense dollars. But I submit this will be a good thing, especially if bona fide experts in different fields join the gold rush as well and start signing their names to articles that they've vetted."
First, I've been saying for a while that someone should fork Wikipedia and start assigning "ownership" of articles to credentialed experts where possible, so that an article can be cited as a source that has been vetted by a recognized individual, and to guard against vandalism. Citizendium does something like this, but started from the ground up rather than fork Wikipedia. I argued that they should fork as much as possible from Wikipedia (having experts "bless" the content in the process); the project's official reason for not doing this was that authors are more motivated when starting with a clean slate than when taking over someone else's article. True, we all know the energizing feeling of a clean slate compared to the sluggish feeling of taking over a 50%-completed project with all of its flaws and compromises, but the "energizing feeling" often doesn't make up for the advantages of having 50% of the work already done for you (which is, in a nutshell, the only reason people ever finish 50%-completed projects instead of starting over!).

So could some other Wikipedia fork achieve the same thing? Programmer/blogger and Guardian columnist Seth Finkelstein, a frequent Wikipedia critic, has pointed out that other sites such as http://veropedia.com/ have tried to build a "verified" version of Wikipedia. "But," he writes, "it doesn't work for many reasons:

1) Maintenance
2) Nobody knows the site exists, or uses it.
3) Google will kill the site's ranking, because of "duplicate content"
4) Roughly 99% of Wikipedia's value is the Google-rank it has, and sites trying to copy its content don't have — or get — that Google-rank.

All true. But Knol has a shot at solving all of these problems. #1 should be mitigated if users earn money for maintaining articles — and besides, many articles like "Abraham Lincoln" won't need much maintenance anyway. #2 should not be an issue since it's a Google project. #3 and #4 depend on how Google lists Knol pages in its search results. The VP's blog post says only,

"Our job in Search Quality will be to rank the knols appropriately when they appear in Google search results. We are quite experienced with ranking web pages, and we feel confident that we will be up to the challenge."

Of course the question on everyone's minds, not answered directly by those sentences, is whether Knol pages will get any special treatment in search rankings. Google would probably be criticized if they manipulated the results outright. But they might achieve the same result indirectly — for example, having a tab across the top of their search results page for "Knol results," along with the tabs for Web, images, and news. Or if Knol results get killed for "duplicate content," Google might (legitimately) consider this a bug and tweak their duplicate-detection algorithm. Thus Knol would have the same advantage in Google that Microsoft's Media Player has on Windows: The operating system doesn't favor Media Player directly, but compatibility problems with Media Player will always get fixed first (while the RealPlayer people have to watch their programs get broken by Windows upgrades). One way or another, it's pretty certain that Knol results are not going to be "unfindable" on Google.

Now, I'm sure Knol will not formally fork Wikipedia. I wouldn't see any problem with them doing that, but it would be too controversial, after the VP announced it without ever mentioning "Wikipedia," and with Google already dealing with speculation that they're only creating Knol to complete with Wikipedia in their own search results. But with users having cash incentives to copy content from Wikipedia, probably most of the content would get replicated very quickly, and I would be surprised if many users didn't start writing scripts to robo-copy as much content from Wikipedia as possible.

Then you get to the point where experts start improving it. If the first couple of entries on "Physics" are just the robo-copied Wikipedia version, "signed" by users that nobody has ever heard of, this is barely an improvement over the unsigned article on Wikipedia itself. But then only one Physics professor in the entire world has to think it's worth their while to read the standard Wikipedia article, make any necessary corrections, and sign their name to it on Knol — and now you have a version that has been vetted by a credentialed expert, increasing its value many times to people who want to cite it as a source, or who want a higher degree of confidence that it's accurate. (Hopefully Knol will allow authors to confirm their e-mail addresses and display them — in an image, presumably, to stop them being scraped by spammers. This will allow professors to prove that they really have faculty .edu addresses and enhance the credibility of their articles, something I suggested for Citizendium.)

So, some criticisms of Wikipedia would not apply to Knol. Author Nick Carr has written of Wikipedia,

"Certainly, it's useful - I regularly consult it to get a quick gloss on a subject. But at a factual level it's unreliable, and the writing is often appalling. I wouldn't depend on it as a source, and I certainly wouldn't recommend it to a student writing a research paper."

When I asked if he would recommend Knol for the same purpose, he was more optimistic:

"Probably. Since a Knol would be written by an identifiable person at an identifiable point in time, I don't see why you wouldn't treat it, in doing research, in a similar way that you'd treat, say, an article by that person. Obviously, you'd need to judge the writer's expertise and authority when deciding whether or not to draw on his or her work, which becomes somewhat more problematic where no editorial or peer-review system applies, as in Knol."

This is where I think the value of a professor's .edu e-mail address comes in, which can at least establish a writer's authority in their subject. I asked Seth Finkelstein whether he would recommend Knol in those same circumstances (verified professor's .edu address, etc.), and his take was, "Of course I would, but you loaded the question in a way so as to remove any problem from it." Well, yeah. I just happen to think Knol actually could remove those problems.

Then there was a little-noticed phrase in Google's blog post that suggests another area where Knol could improve over Wikipedia: the inclusion of "how-to-fix-it instructions." Given that people often need how-to instructions a lot more badly than they need encyclopedia articles, it's surprising that there hasn't been an attempt to standardize around a "Wikipedia of how-to's." Perhaps it's because the Web itself actually does pretty well for that — type in the text of some error message, and you'll usually get some hits on support forums where people ran into the same problem. The trouble is that the ranking of search results depends on the popularity of the site, not on whether the thread ended with someone posting a solution to the problem, so you might have to read through a lot of search results to find an answer. And if you're an expert who happens to know how to do or fix something, there's not much incentive for you to post a page about it (even with AdSense ads), because your page will get buried in the search results beneath all the support forums discussing the same question, even if your post is more concise and useful. Some gurus like Dave Taylor and Leo Notenboom have written so many how-to articles that their own sites have risen through the Google rankings, so if they write a how-to article about something, it will get read (which, of course, creates an incentive for them to write more of them). But for a new expert just starting to write how-to articles, it would take a long time to reach that critical mass.

Knol, however, creates an incentive for experts to start posting how-to-fix-it advice and start reaping the rewards right away, since your how-to articles are just as easy to find under a given subject as anyone else's. Your earnings would start out small as you began to write articles, but they would rise in proportion to the number of articles you wrote, and you wouldn't have to slog along writing for no reward like a typical blogger or site creator, hoping to hit "critical mass" some day. You'd find out early on what the reward would be (financial and otherwise) for the work you were doing, and could decide if you wanted to continue.

Actually, the possibility of "instant rewards" does depend on how Knol articles are ranked against each other within a given topic. The Google blog post says, "For many topics, there will likely be competing knols on the same subject," and, "Knols will include strong community tools. People will be able to submit comments, questions, edits, additional content, and so on. Anyone will be able to rate a knol or write a review of it." Presumably the top-rated articles on a given topic will be displayed first by default, so I'm making the assumption that good articles really will get sorted to the top. But if there are already 50 articles on some topic, even if you know you can write a better one, how do you know it would rise to the top of the pile? If 10 people rate your article a 9, and all the other articles have been rated by 100 people each and got an average rating of a 7, then yours should still be listed as the highest in terms of average rating. But how do you get even those first 10 people to see your article? You could invite your friends, but then how do you stop anyone from gaming the system by inviting their friends and asking them all to rate their article a 10?

I'd written about this in the context of whether Wikia Search might try to solve these problems by allowing users to vote on search results, if they could prevent people from gaming the system. That was basically just me thinking out loud that something like that would be cool, before Wikia Search announced any specifics, and I haven't heard that Wikia is trying anything like that. But now that Google has stated that they will use voting and ranking systems in Knol, the question is how to reward new authors while preventing cheating. I suggested some ideas in an article about how to stop cheating and vote-buying on digg. One idea was that you could have a section on the page that showed people links to different articles at random, so that users couldn't self-select on what articles were shown to them, and if those randomly selected users followed the links and voted on the articles, count only those votes in determining the true "rating" of an article. (This is what HotOrNot does for people's picture ratings.) Even if few people rated those articles, the ratings that were collected, would be representative of real users, and not the horde of friends that you'd sent to rate your article. If that did not prove popular enough, then you could give authors the option to pay random users to rate their articles — as long as there was no way for authors to tie payment to higher ratings, the ratings would average out to reflect the article quality, and could be used to sort articles based on their actual merits.

So, I'd like to think that someone in the Googleplex is reading everything I write, but it's probably just a case where great nerds think alike. I wrote in Feb 07 that I thought Citizendium should allow authors to put their name next to articles, both for the "name up in lights" incentives factor and to enhance the article's credibility, and now Knol is going to do that (not to mention throwing in money as well). The same month I wrote that someone should build a search engine that groups together user-submitted articles under different topics, and provides a means for newly submitted articles to rise through the ranks as a result of user votes, and it sounds like Knol will attempt that too. Then in April 07 I wrote about the ways that you could prevent cheating in such a system, and even though Knol hasn't talked about what they will do to address that problem, they're almost certainly thinking about it, and have probably come up with some of the same ideas.

So let's do a test to find out if Google is reading these articles. There's one area where Wikipedia would beat Knol, and that is that everything on Wikipedia can be redistributed for free. That's something really special, and it's the one part of the Wikipedia hype that I actually buy into. I don't really care that Wikipedia articles were created as part of a "worldwide collaborative effort" unless that helps to achieve the goal of being useful. But Wikipedia, for all its flaws, represents the first time in human history that we have a compendium of a huge amount of human knowledge that can be copied freely, that literally belongs to the world, and because it's duplicated in so many places, it can literally never be taken away. That part of the hype really is true, and is quite heady when you think about it.

Google Knol has not declared this as one of their goals; a Knol article might not be freely distributable. When a proprietary project is hosted on a private site, there's always the risk that the company will pull the plug on it. They probably won't pull off the content offline, but they might shut the service down to stop new content from being added, the way Google did with Google Answers. Yes, Knol authors will retain ownership of their writings, so they could try to regroup and continue the project somewhere else, but it would be a huge mess to try and contact all of the authors and get their permission to copy all of their articles to the new location. As currently planned, Knol doesn't "belong the world," and Google never promised not to take it away.

So, I think that Google Knol should include a feature whereby authors can flag their articles as being freely distributable under the same terms as Wikipedia articles. (Any author who copied an article from Wikipedia and submitted it, would be required to set this flag, because under the terms of "copyleft", you can't copy something that's freely copyable and then try to stop others from copying it!) Then a user who wanted a copylefted, freely distributable article, could limit their search to articles that have this flag set. This would give Knol the best of both worlds: if the author of the top-ranked article did not wish for it to be freely redistributable, then they wouldn't put it on Wikipedia, but they could make it available on Knol, and users could choose either the top-ranked copylefted article or the top-ranked article overall, depending on what they wanted. If the best article on a given subject also happened to be flagged freely distributable, then so much the better.

Maybe the Knol people have had this idea already. But even so, if they end up implementing it, then I'm starting right away on articles about how Google should implement Google Anti-Censorware, Google Site Hijack Prevention, Google Security Compensation, and Google Sergey And Larry Give Bennett Their Airplane.