Google Blocks 'Optimized' Pages
Rhett Creighton writes "For the past few years, webmasters have found tricks that bring their page higher for a given keyphrase search. Google recently implemented a filter to block sites that appeared to be tricking it into gaining a higher ranking. This NYTimes article reports of angry retailers who are losing their businesses, while this article gives more technical conspiracy theories of what google is actually doing."
Seth Finkelstein has been posting a few theories lately on what Google is up to. (Also contains links to other articles.) He suspects they are using some sort of Bayesian filtering around the rule "If a simple search has spam-related keywords, penalize high-spam-scoring results" (spam being search-keyword spam on web pages -- not e-mail spam)
<sig>Guvf vf abg n frperg zrffntr
Google needs to fine tune their code. Enter "goatse.cx" and clicking "I'm Feeling Lucky" brings you to their uptime page at Netcraft, not the horrible image we all know and cherish.
ps: f1st pr0st.
Trolling is a art,
Every month or so, Google updates its database again, and every time, webmasters all over the world whose pages happened to go lower in the rankings complain that Google is broken and the sky is falling. This time is no different, except that mainstream news has picked up the story. Here are a few facts to keep in mind:
1. You can't say with authority that "Google has implemented a filter." Google isn't talking about how their rankings work. The webmasters and SEO types are like astronomers trying to figure out how Google works by observing samples of results. Take everything they say as a theory and nothing more.
2. There's a fine line between making responsibly search-optimized pages and spamming Google, and many of the people who complain are on the spamming side of that line. If you look in the forums where SEO types (and spammers) hang out, 90% of the messages are complaining that their site has disappeared and Google is wrong. If you look in web development forums, 90% of the messages are from people excited to see their pages' position increase.
3. For every webmaster that complains about their site's Google position going down, there are one or more sites whose positions have gone up. Often they're equally deserving of the traffic.
4. There are strong rumors (and some statements from a Google representative) that suggest that this is the last major update to Google's database, and that incremental "freshbot" updates will continue from now on. If this is the case, it may only be a day or a week before your site changes position again, so why complain?
5. Most importantly, notice that it's always webmasters complaining. Never end-users. Guess which group Google considers its customers?
It's Slashdot's evil twin... SlashNOT
With the huge number of postings on all the various forums, concerning this update, most people don't know where to start looking for information about the recent Google update. The following is an attempt to put down rationally (I hope) most of the information that is known and the (unproven) theories behind the update algo.
Introduction.
Starting on the 16th of November, a major shift in results was seen on Google. Veterans recognised that Google appeared to be doing a major update, not seen for many months, as reported first on WebMasterWorld who named it Florida, continuing the tradition of naming updates rather like hurricanes. In this case it was a hurricane! As was usual with many updates, there were moans and groans as people complained about their sites falling. Many people were unaffected (including us) but the symptoms of the sites being dropped were not usual. No penalties, such as PR0, seem to have been applied against pages that had fallen - yet none of the pages targeted at specific key phrases, particularly index/home pages, appeared in the top results for these search terms. Indeed some had dropped hundreds of places and, in some cases reported, off the scale. Yet these pages did appear for obscure phrases and were obviously still in the index.
It appeared to us and to several other respected names (though hotly disputed by others) that some sort of over-SEOd filter had been applied to check if overt SEO had been done for that particular phrase. It was as if Google were checking to see if external links to the site included the phrase, on-page optimisation was being done for the phrase and even if the domain included the phrase. If the density of the optimisation, both on and off the page, appeared too artificial, then a filter was tripped and down went the page - solely for that phrase.
Google had never looked favourably on abuse of their systems and many established SEOs looked upon this algo tweak as a way of Google getting rid of the abuses of links and stopping the scrambling for getting (and sometimes buying) links including your required anchor text from other high PR, but probably irrelevant to your subject, sites. It seemed to make sense.
On Friday, 21st November, Google decided to tighten the filter. All hell broke loose as tens of thousands of sites disappeared from positions they had held (in some cases) for years. We noticed some of our client sites plummeting for their major key phrase from being #1 to total invisibility. Yet this was only in highly competitive areas, not for their secondary phrases. These sites were, in most cases, not highly optimised, had not sought reciprocal links but had achieved their rankings through being on the web for 4 or 5 years. The bad news was that their company name and domain included the key phrase, sites (including directories) linking to those sites included the key phrase in their links and Google interpreted this as over-optimisation and down they plunged. In many areas all the top 20 ranking sites disappeared, including industry leaders, to be replaced by educational sites, news review sites, government sites, major shopping portals or directories. Something major had happened - but what?
The Facts!
Thousands of web pages have been suddenly demoted in the Google search results, primarily on the main commercial search terms for which they targeted their pages to be replaced by other sites who, in the main, referred to the search term obliquely. Several were the main shopping portals or business directories which gave listings for companies who may provide the services requested, many were not.
Very high-ranking authority sites seemed to be unfiltered.
The changes were starkly obvious on regional English language Googles where a regional filter was employed and there were less commercial sites with authority.
An example for Google UK is the search for the word shelving. On the
Socialism: A feeling of discontent and resentment caused by a desire for the possessions or qualities of another.
I for one really don't give a rip if retailers throw a hissy over this. When I search the web it's because I want information, not because I want to buy something.
If I want to buy something I use Froogle. That's what it's there for.
The real moral here is that if you're depending on your placement in a search engine for free advertising, you'd better have a backup plan.
Jon Acheson
All opinions expressed herein are my own, and not those of my employers, who are appalled.
Google was my #1 tool to find my penis enlargement products.
Now I can't even get a home loan!!! And I can't consolidate my debts!!!
What am I gonna do???
I wrote a small software app for a the company I work for, and someone linked to it from a discussion forum.
.exe file.
That gave that name of that program xxxxx.exe 1 hit on google. About 3 days later a search for the xxxxx.exe provided 3 hits, two of them were porn sites that somehow harvested the name of our
While it's not a huge deal, I e-mailed Google and heard nothing for 4 days. I didn't expect a response and told them that in my e-mail. However then I recieved a personalized (not a form) e-mail regarding my comments and that they'd take the issue seriously.
24 hours later they were able to filter out these porn sites that were harvesting new terms that appeared in Google.
I gotta say props out to the boys there, it's one classy establishment.
Google tries to find a formula that gets you the best result for what you're searching for. Some web site owners try to figure out this formula in order to make their page show up in the search resuls when it shouldn't (e.g. by having words in the URL or within certain tags on the page -- rather than by having content relevant to that topic). This makes Google less useful (including for the purpose you describe) and so Google is "demoting" pages which show signs of using these tricks. This tug-of-war has been going on as long as there have been internet search engines. The difference now is that Google accounts for so much of the searches on the net that getting a lower rank in Google can have a huge effect on a site's traffic and so people freak out about it.
<sig>Guvf vf abg n frperg zrffntr
Olagam
Google - thanks abound from me. I personally find it distasteful when I'm searching for research on a particular topic. More times than not - most of the top listings are by an amazon or other shopping portal that has NOTHING to do with my search.
Yes, many businesses are being hurt by GOOGLE's policy - however, it is GOOGLE's search engine! They have done nothing wrong but try and give authentic results to their Web Surfing friends.
Google has a specialized tool to access their search engine specifically to do shopping/price comparison... So yes, you are missing something. :)
Besides, these sites were using hacks to artificially inflate their pagerank instead of providing a higher quality site to increase it.
Maxim: People cannot follow directions.
Increases in truth directly with the length of time spent explaining them
Given the underlying reason why google is a good search engine (leveraging the popularity of the site by others), I don't want "my" search engine to be fooled into giving me commercially-orientated results.
If Google has re-organised the page-ranking system to cut out the link-merchants, I give it an unreserved thumbs up
Simon.
Physicists get Hadrons!
...is no substitute for a business plan.
So some people are trying to cheat the system, and Google is taking steps to prevent this. Good for them, I say. I'm tired of getting pages that appear to be legitimate, only to find that they're just redirect fillers.
As for Google's practices in general, retailers are free to moan and groan about their rankings, but there is no obligation for Google to specifically cater to their needs. If Google decided to change its algorithms, such that all links were turned alphabetically rather than by PageRank, they would be well within their rights to do so. Of course, I imagine that such a move would result in many people seeking other search engines soon enough.
About 50% of the time when I'm searching, I AM looking for vendors of a product in order to do price comparisons
Google already has a search engine specifically designed for price comparisons... maybe that's what you're missing.
It's called Froogle.
I'm not sure why they haven't added a tab for this on their main page, as it would make a lot of sense to separate out commerce-related searches from information-related searches while making both easily accessible.
Maybe partying will help...
Don't they realize that Google wasn't created for them? Rather, it is created for surfers. There is one surefire way to get noted on Google if your business depends on it: advertise. You get what you pay for.
"Common sense" comes to mind.
If you read the NYT article, it clearly says that some unscrupulous vendors were clogging up the search results. So, on the first page of Google results, you'd get most of the sites from the same vendor (shell sites, put up specifically to increase the number of links between them, thereby increasing the PageRank).
Google is trying to level the playing field, so that no one site can dominate the results.
Looking at your complaint, I think it would make sense for Google to have a "vendors" checkbox, which would list sites selling stuff, as opposed to sites giving out information.
for "boost page ranking", and got this cheesy online marketing company:
http://www.page-rank.boost-web-site-traffic.com/
My first reaction was that these guys are scam artists. But, they did appear at the top of the page...
Obviously it is still possible to scam google. To web experts: how do they do it?
"If you think you have things under control, you're not going fast enough." --Mario Andretti
The free search engine that listed me for free is no longer paying off! Waaaaaa! Waaaaaa!
I for one welcome the change. Too many times have 20 of the top 30 links taken you two one site, but camafloged to google somehow as to look seperate. I experieced this painfully while looking for ringtones for my cellphone.
Google is first and foremost a search engine, not a marketing tool. Those who thought otherwise are finding out they are sorely mistaken.
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
I think that'd be cool, or at least have an option for it. Googling for information about things like video cards is almost impossible because all the links are to stores.
Despite my personal pride in being one who tries to grasp a concept/issue via multiple sources from different perspectives, i just realized that the vast majority of my information these days funnels through google. And i know i am not alone.
I would wager google's potential control of information distribution and content filtering rivals that of major centralized information outlets like CNN or the NY times. Kinda unnerving.
From the second linked article:
An example for Google UK is the search for the word "shelving"... On the main Google search for the same phrase, the results return 1 site that sells shelving, 6 shopping portals, 2 Universities and 1 Amazon store. Yet previously these results showed 9 shelving suppliers.
What does this guy expect? He searches on a single word and expects that every result be a retailer? Why not add some extra terms, like "buy" or "seller" or "retail" after that, buddy?
Seriously, should I start crying foul when I search Google for "dog" and it returns information on breeds rather than specific pet-stores?
Who do you get to be an expert to tell you something's not obvious? The least insightful person you can find? -J Roberts
I run a teeny Miva Merchant site which used to be a 'regular' (html, cgi) web store. It has been up since 1997 with little or no changes. No meta tags or keywords on the site.
A week after I messed with that stuff, Google put my site at the top of the list when you do a search for some uncommon keywords related to it. It was nice to see, but so far of limited usefulness.
Now with the Miva site (which are notorious for not being indexed) I will have to come up with a revised strategy.
I would tell anyone - pay attention to your tags, and the immediate content of your site. Everyone is fighting for placement using similar keywords, so checkout the top results and see what they are using.
OT - Doesn't Barry look like he's making some deal on the phone? "Yeah, I can get you Ted, but he's gonna cost ya. He's huge at the Laugh 'n' Snort. I think he'll go for that, I'll call ya back."
As a long time user of Google and a reasonably large advertiser - our company is now questoning whether Google will survive the next couple of years. Through contacts, we have pointed out to Google (and submitted spam reports and submitted poor results reports) that one of our competitors has 2,700 duplicate doorway entry pages to their site. Several hundred of those are illegally indexed using "our" trademarked name. We also advised them of another competitor with 159,000 doorway pages - all indexed and showing up in results. Google's response . . . (silence)
i don't think you did. if you did, you would have known that they gave a prime example of a larger company fouling up the results and really hurting a small company that had been doing fine before the larger company started spamming google.
... how many people do you expect to actually go to the 2nd page?
case in point, last night, i decided i was going to buy the "i'm a bomb technician. if you see me running, try to catch up" shirt. when i searched for it, the first 7 results were more or less the same page (different bgcolors, different rotating ads, different popups), which all pointed to the same url for checkout. i didn't find the vendor i wanted until the 2nd page
i guess ihbt.
vodka, straight up, thank you!
A few days ago I tried a seach for "cellular customer satisfaction". The first several pages were bogus resellers (many of them the same page under different URL's). None of them contained the kind of information I needed about how customers rate the various cellular service providers. This morning the same search is yielding lots of useful data instead of the fake spam-like pages I had been getting.
KUDOS to Google for fixing this! Whatever changes they've made to their pagerank algorithm, Google is suddenly working again like I expect.
Life is short: void the warranty.
Google's "PageRank" formula is their top-secret way of determining in which order to display websites for any given keyword. Everyone knows that refering links is the main component of PageRank, but Google has always been hush-hush as to what else is included in the formula.
It's also known that PageRank isn't a static formula. Google reserves the right to change it at any time, in what is known by Google-watchers as a "Google Dance".
The only legit way to be highly ranked by Google is to be the most authoritative source for information about whatever you discuss, and naturally links will form from other quality websites on your topic and up the PageRank scale you go. www.microsoft.com being a 10/10 ranking doesn't indicate that Google likes Microsoft, it just simply indicates that site is the most authoritative site about a topic a lot of people talk about, Microsoft's products.
Any other way to cheat the system will result in penalties applied to your score. It's not so much a filter as it is negative factors in the formula. Google steadfastly claims that it doesn't maintain a blacklist of "bad" sites, but it is clear that sites designed to cheat Google's PageRank formula always fail once Google tweaks the formula. They don't need a blacklist, they simply identify the characteristics that define a "link farm" and then apply a penality. If a given site has a lot of links to external domains, very little non-link content, and absoulutely every linked to site returns a link back to the orignal site, it sure smells like a link farm and that's what the system penalizes.
To put it bluntly, anybody who's business depends on being displayed on the first page of Google results should be buying AdWords placements. If you're working hard to stay #1 in the editorial results, you're never gonna win. And no, just because your business depends on it doesn't mean you get to sue when Google pulls an ill-gotten #1 ranking out from under you.
Google is a service that is, for the most part, free to those who benefit from it.
Somebody discovers that they can manipulate this service to increase their benefit.
The people who provide this (free) service chose to ignore those manipulations. Maybe they deliberately lower the ranking of some pages, to hear the whiny TFH crowd speak.
Then those same whiners--who contributed NOTHING to the process from which they benefit--scream for damages.
If someone invented a pill to make people immortal and one of these jerks didn't get his pill, these same folks would want the inventor jailed for murder.
Until you form a union and negotiate a contract with google--that includes a "past practices" clause, just STFU.
"Reality is that which, when you stop believing in it, it doesn't go away." - Philip K. Dick
Right, but when you search through Google (or any search engine) you expect to find rankings that accurately reflect the relevance of the page. If you search on "testosterone" you probably do not want the first ten pages of links to be to "Joe's Patented Penis Enlargement System" just because hardworking Joe set up dozens of shell web sites solely to increase his site's Google rank.
If Joe (or any other web site owner) really wants to use Google as an advertising medium, he ought to pay for a sponsored link and be done with it. Joe has no right to manipulate the ranking system, and if he's going to do that he ought to be prepared to suffer the consequences.
>
> Not really. "I'm feeling lucky" that I, unlike the gentleman in the picture, do not have a 6" diameter asshole.
At least! I know what to give thanks for tomorrow at the family dinner! Dude, Thanks!
Read the article. The company set up dozens of websites in order to get themselves in the top ranks of google - as in, all of the top 10 that you would see on google would in actual fact *be the same company*.
That acn hardly be considered a situation in which you can price-compare. Google is simply fixing the problem.
That's why I have to laugh whenever I read stories speculating that Microsoft might do to Google what they did to Netscape. It's one thing to steal a big consulting/integration contract by throwing lots of marketing and engineering resources at the customer. But to dominate the search engine world, you have to earn and maintain the trust of millions of users who pound on your engine every single minute. I used to think that Infoseek, Altavista, and the others died solely from corporate neglect. That's partially true, but they were doomed anyway, as soon as Google appeared. Because none of them ever understood what Brin and company seem to understand instinctively -- a public search engine requires hard work on a huge scale, and it never stops.