Slashdot Mirror
El Reg Says Google Choking on Spam Sites
Grubby Games writes "The Register is reporting that Google is full, and in trouble." From the article: "Recently, we featured a software tool that can create 100 Blogger weblogs in 24 minutes, called Blog Mass Installer. A subterranean industry of sites providing 'private label articles,' or PLAs exists to flesh out 'content' for these freshly minted sites. And as a result, legitimate sites are often caught in the cross fire. But the new algorithms may not be solely to blame. Google's chief executive Eric Schmidt has hinted at another reason for the recent chaos. In Google's earnings conference call last month, Schmidt was frank about the extent of the problem. 'Those machines are full,' he said. 'We have a huge machine crisis.'" James Robertson points out that's a fairly selective bit of quoting.
Thanks!
This issue is a bit more complicated than you think.
With hardware (and bandwidth) getting cheaper, I find it hard to believe that Google has actually run out of space. But certainly the explosion in the number of web pages is an issue, especially with auto-generated pages. One current example is the V7ndotcom Elursrebmem SEO contest (white-hat celiac charity site I'm supporting) - that nonsense phrase returned zero results on January 15th, 2006 ...
but now returns almost 5,000,000 ... of which I gotta believe the
vast majority were NOT typed in by humans.
So maybe it's more that the techniques/algorithms used to spider and index are struggling with the bazillions of web pages out there. Or it could just be disgruntled webmasters PO'ed that their web site isn't listed!
Hulk SMASH Celiac Disease
Wow...so there really is an end to the internet.
concrete5: a cms made for marketing, but strong enough for geeks.
I'm not a computer person, but couldn't Google just upgrade to a bigger disk drive?
I saw one at bestbuy.com that looks pretty good.
In creating adsense, google opened the floodgates for spammers who do not want to create good content. In fact, there are even people who copy tons of content from wikipedia and throw up adsense on the top and sides of the pages.
There are people who are literally making $10,000 or more per month just putting up junk content sites that are auto generated for the purpose of creating adsense revenue.
Don't get me wrong, I think adsense is a good thing, but Google's allowance of spam sites is giving adsense a bad name.
Meanwhile, for no good reason, here's some gorgeous stats porn on how Google (and Yahoo and MSN) crawled a sample website. The animations and closeups of the trees are very cool.
A stingy old spammer who had been diagnosed with a terminal illness was determined to prove wrong the saying, "You can't take it with you." After much thought and consideration, the old spammer finally figured out how to take at least some of his money with him when he died. He instructed his wife to go to the bank and withdraw enough money to fill two pillow cases. He then directed her to take the bags of money to the attic and leave them directly above his bed. His plan: When he passed away, he would reach out and grab the bags on his way to heaven. Several weeks after the funeral, the deceased spammer's wife, up in the attic cleaning, came upon the two forgotten pillow cases stuffed with cash. "Oh, that darned old fool," she exclaimed. "I knew he should have had me put the money in the basement."
Over the past 6 months or so, I've been finding a lot of link farms in my search results. Oh, irony or irony, SEOs are making search results worthless.
Some of you might recall that for a long time the Google index stood at around 4 billion pages. It turned out that this was because of the limited number of unique 32 bit index values. To handle this, Google created two index values to reference each each page. One is called the "Selector", and the other is called the "Offset". Simply put, the selector is left shifted by 4 bits and added to the offset so that Google can find any page on the internet simply by knowing its selector and offset. According to the article. Google has exhausted these values as well, and will introduce something called "protected mode page rank" where the slector is shifted farther to create a greater range of values.
Unknown host pong.
Try this...
i es
Go to yahoo and search for "slashdot poneys". This will bring up a bunch of results, all approximately 1 month old.
Now do the same search on google. Notice how many of the results from yahoo do not appear in the google results at all.
Google has such a big backlog that they don't get around to spidering new sites for several months. While google does give priority to certain high-profile sites like slashdot and visits those frequently, most other sites do not get indexed for several months.
Okay, so I tried this, just for kicks. You can verify, by a single click:
Yahoo: http://search.yahoo.com/search?p=slashdot+ponies
Google: http://www.google.com/search?hl=en&q=slashdot+pon
Since when does 44900 results on Yahoo mean that they have more than 92100 results on Google? As far as what's appearing, I was able to find most every one I saw on Yahoo on the first 2 or so pages of Google's results. I also see more results on Google that look like they'll show me more of what I'm looking for (since I am probably looking for the April 1st joke, screenshots especially).
Works alright for me. Looks like I don't have a reason to switch again yet.
Well given that a human would have a hard time deciding if the page was autogen'ed if the text was in their second language, this *is* quite an issue.
So it sounds like Google needs to *shudder* have a user feedback system where humans with logins add moderation metadata to the search results and in return get results based on this moderation en-mass.
I know what your thinking,
It would withstand abuse since a massive amount of human inputed data would keep spambots from trying to exploit the moderation system. What's more, their toolbar could incorporate the control to flag a page as autogen'ed garbage.
You are checking your backups, aren't you?
...then eventually the spam sites will actually contain the information you were looking for.
44 on yahoo, 229 on google.
Wait, what was I saying?
Man, you really need that seminar!