Modern Day Search Engine Manipulations

An anonymous reader writes "I fondly recall the days of yore when search engines could be manipulated just by sticking thousands of extraneous filler words in the META tags or hidden at the bottom of the page. Nowadays search engines work by more advanced techniques that generally don't fall prey to these simplistic tactics, but it'd be folly to presume them impervious. Does it still happen?"

Comments

yep

[twiggy]

Yes, it still happens a lot... there's widespread knowledge of so-called "google bombing".. Google pops up some of its search results based on the content between an A HREF tag, as you can read about here: Google Time Bomb...

Much like security, I think this is the kind of thing that hackers and tinkerers will always find a way to exploit. The question is who can stay ahead in the race?

the new status quo

[Greg@RageNet]

The new status quo for search engines seems to be to charge for submission, as many of them now require you to go through a third-party that charges to add your site to the database. The variation of that (ie yahoo) has 'sponsored' sites in each category that appear at the top of the page. A friend runs a site that uses this 'sponsored' system and I'm told those sponsors bid against each other and whoever has the highest bid appears.. kinda like an EBAY for search engines.

-- Greg

Re:the new status quo

[Evro]

Yahoo does not charge for submission, but you'll likely never make it into their db either, because everyone submits. If you pay them $200 then you're guaranteed that they will review your site within 2 weeks, though this does not guarantee you'll be in their directory.

It's also worthwhile to mention that Yahoo's not really a search engine in the sense of something that crawls the internet looking for info; they generally rely on submissions, with which they're surely inundated, and that tiny subset of the internet is what they search.

As for sponsored links, 75% of the "sponsored links" on search engines are culled from Overture (formerly goto.com). Goto took a lot of heat back in the day for selling search results, but they've found a market in selling these results to other engines. Until like 3 or 4 months ago, their results were on Yahoo, AOL, Netscape, Altavista, and most other search engines. Then Google got into the bid-for-keywords market with their Adwords Select program. Now in addition to searches on google.com, Google's adwords show up on searches on AOL, Earthlink, and a few others. The process is basically as you described - bidding for keywords. Usually it's not worth bothering unless you're in the top 3 for that keyword on Overture, as those are the ones that show up on Yahoo (I think #4 and #5 show up at the bottom of the page). On Google I've seen up to 8 ads for a given keyword (e.g. computers) but AOL only takes the top 3 for its "sponsored matches" as well.

On Google it's important to note that the sponsored sites and the real search results are completely separate (dependent on how much you trust google, of course, but they have a lot of karma built up), and google's results are gleaned from having their robot (Googlebot) crawl the web, not from submissions; and the algorithm that ranks sites is another matter entirely. E.g. a search for "ass grabbing computers" predictably has 0 results, but there are plenty of ads for the word 'computer' that pop up.

It's doubly important to note the above about google since many Yahoo searches fall through to google when there aren't any results in yahoo's (IMO Lame) directory, so the results from yahoo are not as paid-for as you seem to imply.

who cares?

[reaper20]

Nine times out of ten, when using Google, exactly what I am looking for is in one of the first few links.

I had a boss that was asking me "How do we improve our site on google?"

Answer: Provide actual information instead of some glossy maketrdroid garbage that is so prevalent in webpages today and you wouldn't have to worry about the search engines would you?

Re:who cares?

[Frank of Earth]

True. But you can have the most related site and like the article states, unless the domain or pages match the content, most likely, you will not rank high.

Let's say you had the best article in the world about installed redhat, but the link was to www.fperkins.com/tip.cgi?101

Forget about it, you just won't get linked in the top 10. A good trick is to have your dynamic content create a static page which is, of course, dynamically created from the database. Then you would get something simliar to what allrecipes.com does.

Ie their recipe for "African Chicken Soup" is not recipe_view.asp?id=100 but rather http://chicken.allrecipes.com/az/africanchickenste w.asp Not a great example, but you can understand my example, imagine something like "chicken recipe" etc.

Smart. Notice how they even have a subdomain to chicken.allrecipes.com which can be setup really easily for most sites, especially those that can alias any subdomain to the main domain.

Regardless, getting ranked in the top20 in Search Engines is some skill and knowledge and a lot of luck.

this trick works every time

[Dr. Awktagon]

Here's one I use all the time.. just follow these easy steps:

1. Create a well-designed, easy-to-use web site that follows accessibility and useability guidelines.
2. Fill the web site with useful, relevant information on a selection of topics.
3. Make sure the information is kept up to date, and don't let it become stale.
4. Allow this web site to become popular and authoritative, so lots of people link to it and reference it.

Now, watch your Google ranking rise to the top! IT'S THAT EASY! And you'll laugh all the way to the bank!

Re:this trick works every time

[woogieoogieboogie]

Create a well-designed, easy-to-use web site that follows accessibility and useability guidelines.

You forgot to say "make sure it works in lynx because all disabled people use lynx as their browser."

Who makes the "guidelines" for usability. For accesibility? Do all disabled people get lumped together so that one guideline fit's all? Each disabled person has their own difficulties and there is no one size fits all approach. Disabled people are no different that any other person and it is up to them to empower themselves with the technology to view any webpage regardless of guidelines used.

Maybe we can use the gubment's guidelines and use PDF files which rate along with Flash as major web annoyances. I mean, so what if a disabled person gets annoyed having thir computer freeze because some clueless moron decided that the best way to give out a 1 page brochure was to put it into a 2 mb PDF. Don't you think disabled peopel get annoyed at this crap also. But it's okay, because it fit's the disability guidelline.

The best guideline any web developer can use is both common sense and do not interfere with the user regardless if they are disabled or not.

Re:this trick works every time

[Suppafly]

Create a well-designed, easy-to-use web site that follows accessibility and useability guidelines.

Fill the web site with useful, relevant information on a selection of topics.

Make sure the information is kept up to date, and don't let it become stale.

Allow this web site to become popular and authoritative, so lots of people link to it and reference it.

?????

Profit!!!

The Britney Spears mystery

[Otter]

I'm done with work, it's 100 outside and I don't have AC at home so staying late to address the Britney Spears / Shavlik mystery seems like an attractive option...

The relevant bit on one of the Britney Spears pages seems to be:

<IMG src="http://sm6.sitemeter.com/meter.asp?site=sm6bs review&refer=http%3A//www.google.com/search%3Fhl%3 Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26q%3Dlink%2 53Awww.shavlik.com%26btnG%3DGoogle+Search&hours=19 &minutes=59&rtype=1" border=0 title="Site Meter"></A>

Which, yeah, seems to be a roundabout bit of Google bombing.

The question is -- how does this help Shavlik? Presumably there aren't that many people searching for Britney Spears content who say, "Oooh, a way to push Windows patches through a network! I want that!" You'd think the Google algorithm would weight links according to their relevance to the search criteria.

not right

[danny]

Google PageRank (and the search rankings, whch are different to that) are calculated per page, not per-site, so links on pages "in the wilderness" on obscure parts of AOL or Geocities don't count for much.

There may be some confusion because the Google Toolbar, when viewing a page that hasn't been indexed, tries to "guess" what it's PageRank would be based on the site PageRank... but that's not "real".

If you want to know more about Google, the place to go is the Webmaster World Google forum.

Danny.

eBags

[Ken Treis]

While searching for a new diaper bag (the cheap ones only seem to last through 1 kid), I was amazed at how many Google search hits pointed back to eBags. You wouldn't always know it from the URLs, though. Some of the URLs were things like ebags-discount.com, bagsdirect.com, handbags.com, etc., making you think that there were several big bag retailers out there. Others were just plain insane; I remember one that was something like "best-basketball-bags-for-women-athletes.com".

Effectively, they circumvented Google's "site grouping" wherein all hits from one site get clustered under a smaller group. I got fed up with it and resolved not to buy anything from eBags.

But I thought to myself, "maybe they're Scientologists..."

No! Wrong!

[fm6]

That's absurd. Next you'll be telling us that we can raise our /. karma by writing posts that people actually enjoy reading! PUTTING CRAP ON THE INTERNET IS A FUNDAMENTAL RIGHT!!!

Not that I should admit to this...

[Latent IT]

But do a google search for crack/serial/warez.

For instance. Webcam32 Crack

Yes, I OWN webcam32. So there. ;p

The point is, the first THREE PAGES are .de spoofed pr0n pages. Someone figured it out.

Google Limitations

[Evro]

Well this is less so when one accounts for Google's limitations. The biggest of these, in my experience (as someone who works for a site whose google rank directly affects sales) is the fact that Google apparently rarely indexes URLs that contain 3 or more CGI parameters after the "?" character.

For example, a search on google for "plaid socks" yields only 1 or 2 sites out of 100 that have 3 or more CGI parameters, when I'm sure there are many sites using very complicated urls (with session IDs, etc). Sure, this is just anecdotal evidence, but as someone whose product catalog was listed by urls that had at least 3 CGI parameters (and sometimes 5 or 6 depending on the referring URL) I can say with 90% confidence that having a "complicated" URL severely hurt us. What I ended up doing recently was using mod_rewrite to change all the listed URLs on our site from site.com/product.cgi?sku=something&section=2&style =4 to site.com/product/2/4/something.html, and lo and behold, the next time googlebot came by, those pages were indexed (I had verified that the problem was not that the pages had a low pagerank, but that they were not even being spidered at all).

What does this have to do with Google's relevance? Sure, they are returning relevant results when you search, but if they are arbitrarily not listing a site because its URL structure is too "complex" then there's a ton of possibly relevant content that they're missing. If you're someone who sells plaid socks for $10 less than your nearest competitor but Google isn't indexing your plaid socks page because of URL structure (exactly what was happening to us, except not for plaid socks) then you're really not getting the most relevant results. Which is not to say that what you DO see isn't relevant, it's just that there's possibly MORE relevant stuff that you won't ever see.

Fortunately Google has something in the works to cover this particular situation, but it doesn't really have anything to do with fixing their URL complexity policy.

More background reading

[Quixote]

I'm glad people are taking a closer look at Google's ranking algorithm. Hopefully, the scrutiny will make it more robust and tamper-proof.
Here are some more URLs that might be of interest:

• PageRank explained
• Google's Pagerank and how to make the most of it

You got fired, right?

[Pac]

Such an unbelivable display of ignorance on energising the synergies while leveraging the brand-awareness among the propesct client base shouldn't go unpunished.

When my websites needed to be ranked high...

[golemite]

I always check out SearchEngineForums.com for the latest advice. Ranked #4 for Audi S4 and #1, 3, 8-sorta, and 10 for my name ;)

Easy.

[Wolfier]

What do pigeons like?

Put a META tag containing the follow words:
grain, rice, corn, worms, wheat - worked like a charm. You get the idea.

HUMANS do it better...

[Etcetera]

Just a shameless plug here for the Open Directory Project. Leaving aside occasional occurances of editor-fraud or editor-abuse (which are quickly tracked down by the meta-editors), this is the best way to determine a site's real value.

A human looking at the page to subjectively/objectively determine its value is something that can't be replaced by a spider and an AI program.

URL cloaking, hidden text, keyword tricks, etc... don't matter. =)

-jc

Re:HUMANS do it better...

[Backov]

Humans would do it better...

If humans ever got around to doing it.

I know MANY webmasters still waiting for the sites to be reviewed, months later.

Cheers,
Backov

Google Bombing

[Robotech_Master]

One form of Google manipulation that recently hit the scene is known as Google bombing--to wit, getting a lot of people to link to a particular site with certain key words. It was done a lot with blogging, as the article indicates: by linking to a certain artist's page using the words "talentless hack," they caused that artist's page to come up first when one typed "talentless hack" into the search engine.

YAGA - Yet Another Google Article

[meehawl]

Slashdot definitely needs a Google icon.