Modern Day Search Engine Manipulations

An anonymous reader writes "I fondly recall the days of yore when search engines could be manipulated just by sticking thousands of extraneous filler words in the META tags or hidden at the bottom of the page. Nowadays search engines work by more advanced techniques that generally don't fall prey to these simplistic tactics, but it'd be folly to presume them impervious. Does it still happen?"

Comments

yep

[twiggy]

Yes, it still happens a lot... there's widespread knowledge of so-called "google bombing".. Google pops up some of its search results based on the content between an A HREF tag, as you can read about here: Google Time Bomb...

Much like security, I think this is the kind of thing that hackers and tinkerers will always find a way to exploit. The question is who can stay ahead in the race?

the new status quo

[Greg@RageNet]

The new status quo for search engines seems to be to charge for submission, as many of them now require you to go through a third-party that charges to add your site to the database. The variation of that (ie yahoo) has 'sponsored' sites in each category that appear at the top of the page. A friend runs a site that uses this 'sponsored' system and I'm told those sponsors bid against each other and whoever has the highest bid appears.. kinda like an EBAY for search engines.

-- Greg

Re:the new status quo

[Evro]

Yahoo does not charge for submission, but you'll likely never make it into their db either, because everyone submits. If you pay them $200 then you're guaranteed that they will review your site within 2 weeks, though this does not guarantee you'll be in their directory.

It's also worthwhile to mention that Yahoo's not really a search engine in the sense of something that crawls the internet looking for info; they generally rely on submissions, with which they're surely inundated, and that tiny subset of the internet is what they search.

As for sponsored links, 75% of the "sponsored links" on search engines are culled from Overture (formerly goto.com). Goto took a lot of heat back in the day for selling search results, but they've found a market in selling these results to other engines. Until like 3 or 4 months ago, their results were on Yahoo, AOL, Netscape, Altavista, and most other search engines. Then Google got into the bid-for-keywords market with their Adwords Select program. Now in addition to searches on google.com, Google's adwords show up on searches on AOL, Earthlink, and a few others. The process is basically as you described - bidding for keywords. Usually it's not worth bothering unless you're in the top 3 for that keyword on Overture, as those are the ones that show up on Yahoo (I think #4 and #5 show up at the bottom of the page). On Google I've seen up to 8 ads for a given keyword (e.g. computers) but AOL only takes the top 3 for its "sponsored matches" as well.

On Google it's important to note that the sponsored sites and the real search results are completely separate (dependent on how much you trust google, of course, but they have a lot of karma built up), and google's results are gleaned from having their robot (Googlebot) crawl the web, not from submissions; and the algorithm that ranks sites is another matter entirely. E.g. a search for "ass grabbing computers" predictably has 0 results, but there are plenty of ads for the word 'computer' that pop up.

It's doubly important to note the above about google since many Yahoo searches fall through to google when there aren't any results in yahoo's (IMO Lame) directory, so the results from yahoo are not as paid-for as you seem to imply.

who cares?

[reaper20]

Nine times out of ten, when using Google, exactly what I am looking for is in one of the first few links.

I had a boss that was asking me "How do we improve our site on google?"

Answer: Provide actual information instead of some glossy maketrdroid garbage that is so prevalent in webpages today and you wouldn't have to worry about the search engines would you?

this trick works every time

[Dr. Awktagon]

Here's one I use all the time.. just follow these easy steps:

1. Create a well-designed, easy-to-use web site that follows accessibility and useability guidelines.
2. Fill the web site with useful, relevant information on a selection of topics.
3. Make sure the information is kept up to date, and don't let it become stale.
4. Allow this web site to become popular and authoritative, so lots of people link to it and reference it.

Now, watch your Google ranking rise to the top! IT'S THAT EASY! And you'll laugh all the way to the bank!

Re:this trick works every time

[Suppafly]

Create a well-designed, easy-to-use web site that follows accessibility and useability guidelines.

Fill the web site with useful, relevant information on a selection of topics.

Make sure the information is kept up to date, and don't let it become stale.

Allow this web site to become popular and authoritative, so lots of people link to it and reference it.

?????

Profit!!!

No! Wrong!

[fm6]

That's absurd. Next you'll be telling us that we can raise our /. karma by writing posts that people actually enjoy reading! PUTTING CRAP ON THE INTERNET IS A FUNDAMENTAL RIGHT!!!

Not that I should admit to this...

[Latent IT]

But do a google search for crack/serial/warez.

For instance. Webcam32 Crack

Yes, I OWN webcam32. So there. ;p

The point is, the first THREE PAGES are .de spoofed pr0n pages. Someone figured it out.

Google Limitations

[Evro]

Well this is less so when one accounts for Google's limitations. The biggest of these, in my experience (as someone who works for a site whose google rank directly affects sales) is the fact that Google apparently rarely indexes URLs that contain 3 or more CGI parameters after the "?" character.

For example, a search on google for "plaid socks" yields only 1 or 2 sites out of 100 that have 3 or more CGI parameters, when I'm sure there are many sites using very complicated urls (with session IDs, etc). Sure, this is just anecdotal evidence, but as someone whose product catalog was listed by urls that had at least 3 CGI parameters (and sometimes 5 or 6 depending on the referring URL) I can say with 90% confidence that having a "complicated" URL severely hurt us. What I ended up doing recently was using mod_rewrite to change all the listed URLs on our site from site.com/product.cgi?sku=something&section=2&style =4 to site.com/product/2/4/something.html, and lo and behold, the next time googlebot came by, those pages were indexed (I had verified that the problem was not that the pages had a low pagerank, but that they were not even being spidered at all).

What does this have to do with Google's relevance? Sure, they are returning relevant results when you search, but if they are arbitrarily not listing a site because its URL structure is too "complex" then there's a ton of possibly relevant content that they're missing. If you're someone who sells plaid socks for $10 less than your nearest competitor but Google isn't indexing your plaid socks page because of URL structure (exactly what was happening to us, except not for plaid socks) then you're really not getting the most relevant results. Which is not to say that what you DO see isn't relevant, it's just that there's possibly MORE relevant stuff that you won't ever see.

Fortunately Google has something in the works to cover this particular situation, but it doesn't really have anything to do with fixing their URL complexity policy.

More background reading

[Quixote]

I'm glad people are taking a closer look at Google's ranking algorithm. Hopefully, the scrutiny will make it more robust and tamper-proof.
Here are some more URLs that might be of interest:

• PageRank explained
• Google's Pagerank and how to make the most of it

Easy.

[Wolfier]

What do pigeons like?

Put a META tag containing the follow words:
grain, rice, corn, worms, wheat - worked like a charm. You get the idea.

HUMANS do it better...

[Etcetera]

Just a shameless plug here for the Open Directory Project. Leaving aside occasional occurances of editor-fraud or editor-abuse (which are quickly tracked down by the meta-editors), this is the best way to determine a site's real value.

A human looking at the page to subjectively/objectively determine its value is something that can't be replaced by a spider and an AI program.

URL cloaking, hidden text, keyword tricks, etc... don't matter. =)

-jc

Re:HUMANS do it better...

[Backov]

Humans would do it better...

If humans ever got around to doing it.

I know MANY webmasters still waiting for the sites to be reviewed, months later.

Cheers,
Backov

Google Bombing

[Robotech_Master]

One form of Google manipulation that recently hit the scene is known as Google bombing--to wit, getting a lot of people to link to a particular site with certain key words. It was done a lot with blogging, as the article indicates: by linking to a certain artist's page using the words "talentless hack," they caused that artist's page to come up first when one typed "talentless hack" into the search engine.