Google Blocks 'Optimized' Pages

Rhett Creighton writes "For the past few years, webmasters have found tricks that bring their page higher for a given keyphrase search. Google recently implemented a filter to block sites that appeared to be tricking it into gaining a higher ranking. This NYTimes article reports of angry retailers who are losing their businesses, while this article gives more technical conspiracy theories of what google is actually doing."

Comments

Seth F's theories

[turg]

Seth Finkelstein has been posting a few theories lately on what Google is up to. (Also contains links to other articles.) He suspects they are using some sort of Bayesian filtering around the rule "If a simple search has spam-related keywords, penalize high-spam-scoring results" (spam being search-keyword spam on web pages -- not e-mail spam)

I LIKE GOOGLE. :)

[wo1verin3]

I wrote a small software app for a the company I work for, and someone linked to it from a discussion forum.

That gave that name of that program xxxxx.exe 1 hit on google. About 3 days later a search for the xxxxx.exe provided 3 hits, two of them were porn sites that somehow harvested the name of our .exe file.

While it's not a huge deal, I e-mailed Google and heard nothing for 4 days. I didn't expect a response and told them that in my e-mail. However then I recieved a personalized (not a form) e-mail regarding my comments and that they'd take the issue seriously.

24 hours later they were able to filter out these porn sites that were harvesting new terms that appeared in Google.

I gotta say props out to the boys there, it's one classy establishment.

Example of what google is trying to prevent

[Anonymous Coward]

Olagam

Waaaaaa

[scorp1us]

The free search engine that listed me for free is no longer paying off! Waaaaaa! Waaaaaa!

I for one welcome the change. Too many times have 20 of the top 30 links taken you two one site, but camafloged to google somehow as to look seperate. I experieced this painfully while looking for ringtones for my cellphone.

Google is first and foremost a search engine, not a marketing tool. Those who thought otherwise are finding out they are sorely mistaken.

Re:Waaaaaa

[jafac]

Google is first and foremost a search engine, not a marketing tool. Those who thought otherwise are finding out they are sorely mistaken.

That said:
having been a net surfer since 1993, and having NEVER EVER EVER paid money for an online service, other than my ISP -
I would pay Google a subscription fee monthly, maybe even something like $5, to ENSURE that it remained a tool to serve web users, instead of a tool to serve site operators.

Google is THAT important to me.

Without Google, I believe that the web, as a resource for information and communication, would become pointless within the space of 5 years.

I still don't think I'd pay money for any other site. (Including slashdot). Many many sites have started free, then initiated a "premium service" or subscription fee - and faded into obscurity. I think that only Google could survive such a move.

Re:The sky is NOT falling.

[Stephen]

Every month or so, Google updates its database again, and every time, webmasters all over the world whose pages happened to go lower in the rankings complain that Google is broken and the sky is falling. This time is no different.

I agree it's no different from usual, in some ways. But on the other hand, there have been an increasing number of such stories recently. Of course, the loudest complaints come from those whose sites have sunk. But many ordinary people feel that Google search results have worsened gradually over the last few months, and the press is right to pick up on this.

My personal theory, albeit based on no hard evidence, is that Google has started to rely less on PageRank. Newspaper articles usually blame people subverting PageRank in some way, but I think they're wrong. Those papers don't realise that Google uses a combination of many methods to rank pages, not just PageRank.

What I think happened is this:

1. Google invented PageRank. It's not perfect, but it worked much better than anything else, and they took over the world. It's also really hard to thwart: just linking to your own sites doesn't improve your PageRank; you need high-quality links from third parties.
2. Unfortunately, PageRank didn't work so well when blogs came along, because of their high amount of interlinking. So Google was forced to reduce the weight of PageRank in the algorithm.
3. The problem is that this meant they were now depending more on traditional ("SEO") metrics, which are more easily manipulated.
4. So now they're chasing round trying to catch the cheats, while allowing legitimate sites through. But as the history of search engines before Google shows, this is a losing battle, and there will be complaints from site owners who have been wrongly demoted, as well as from users who are seeing more crud.

As I say, all this is speculation, but it makes sense to me.

Re:The sky is NOT falling.

[Webmoth]

"Guess which group Google considers its customers?"

It's not webmasters. It's not end-users. It's advertisers.

Advertisers are the only ones that Google has to answer to. If they do something that makes their advertisers go away, you can bet that they will quickly reverse that decision. And, the only thing that will make advertisers go away is whatever makes end-users go away.

You see, for Google (of for any other media outlet, for that matter) the advertiser is the customer, and the end-user or reader is the product. The content is the means of delivering the product to the customer.

What defines "customer?" Someone who gives you money in exchange for goods and services.

Google does have customers other than advertisers: select webmasters who purchase Google's services for their own Intra-/Internet presences. Even here, the customer is not the end-user, but is the webmaster himself. In this case, Google's best interest is to return searches the webmaster considers favorable (which, ultimately, are those pages the webmaster thinks the end-user should see).

So, you see, Google's interests are where the money is. And the money is in advertising and select webmasters. Perhaps in Google's Internet search, they favor companies who have purchased their services. Perhaps they demote companies who have refused to, but that's only speculation on my part.

Re:The sky is NOT falling.

[czei]

While most months you would be right, in Nov. 2003 you are not. The update to Google this particular month featured a radical change in the algorithms used to rank sites. In previous months for the past several years the rankings would change little every month-- if you look at the top 10 sites for a particular search term you'd see some sites move up a few levels, and other sites move down a couple of rankings.

This time, however, from what I can estimate most of the topped ranked pages have changed for a particular search term, which indicates a major change in Google's algorithm, which is particularly newsworthy. If that isn't news, then what is?

The reason businesses are complaining is that there are tens of thousands of small businesses that make their living from customers who find them via Google, which is also newsworthy. And most aren't SEO professionals or scammers who've illictly tried to artificially boost their rankings, thank you very much.

Whether the change will also elicit complaints from searchers has yet to be seen-- some search terms seem to return relevant results and some do not.

Barry Llyod - Agent to the Stars

[teamhasnoi]

Google's results seem to have degraded over the last couple of years. Full of linkfarm websites and searches that seem to lead to information, but only return another (sponsored) 'search'. It has made it a pain to find common information as the results are spammed by Amazon affiliates, and hard to find obscure information as the results are often buried under a pile of non-appropriate links.

I run a teeny Miva Merchant site which used to be a 'regular' (html, cgi) web store. It has been up since 1997 with little or no changes. No meta tags or keywords on the site.

A week after I messed with that stuff, Google put my site at the top of the list when you do a search for some uncommon keywords related to it. It was nice to see, but so far of limited usefulness.

Now with the Miva site (which are notorious for not being indexed) I will have to come up with a revised strategy.

I would tell anyone - pay attention to your tags, and the immediate content of your site. Everyone is fighting for placement using similar keywords, so checkout the top results and see what they are using.

OT - Doesn't Barry look like he's making some deal on the phone? "Yeah, I can get you Ted, but he's gonna cost ya. He's huge at the Laugh 'n' Snort. I think he'll go for that, I'll call ya back."

Google just doesn't care.

[actionfront]

As a long time user of Google and a reasonably large advertiser - our company is now questoning whether Google will survive the next couple of years. Through contacts, we have pointed out to Google (and submitted spam reports and submitted poor results reports) that one of our competitors has 2,700 duplicate doorway entry pages to their site. Several hundred of those are illegally indexed using "our" trademarked name. We also advised them of another competitor with 159,000 doorway pages - all indexed and showing up in results. Google's response . . . (silence)

Re:What am I missing?

[YouHaveSnail]

Right, but when you search through Google (or any search engine) you expect to find rankings that accurately reflect the relevance of the page. If you search on "testosterone" you probably do not want the first ten pages of links to be to "Joe's Patented Penis Enlargement System" just because hardworking Joe set up dozens of shell web sites solely to increase his site's Google rank.

If Joe (or any other web site owner) really wants to use Google as an advertising medium, he ought to pay for a sponsored link and be done with it. Joe has no right to manipulate the ranking system, and if he's going to do that he ought to be prepared to suffer the consequences.

Bayes Wars

[fm6]

Easy to defeat a bayesian filter: use a sentence generator.

Easy for you. I doubt if Ms. Gifts Baskets knows what a Bayesian filter is. This is an arms race between the index spammers and Google computer scientists -- and Google has a lot of really good CS people. Perhaps somebody smart enough could start a business defeating Googles filters, and distribute the costs among spammer clients. But it'd be cheaper just to buy a Google sponsored link.

That's why I have to laugh whenever I read stories speculating that Microsoft might do to Google what they did to Netscape. It's one thing to steal a big consulting/integration contract by throwing lots of marketing and engineering resources at the customer. But to dominate the search engine world, you have to earn and maintain the trust of millions of users who pound on your engine every single minute. I used to think that Infoseek, Altavista, and the others died solely from corporate neglect. That's partially true, but they were doomed anyway, as soon as Google appeared. Because none of them ever understood what Brin and company seem to understand instinctively -- a public search engine requires hard work on a huge scale, and it never stops.