Slashdot Mirror
Modern Day Search Engine Manipulations
An anonymous reader writes "I fondly recall the days of yore when search engines could be manipulated just by sticking thousands of extraneous filler words in the META tags or hidden at the bottom of the page. Nowadays search engines work by more advanced techniques that generally don't fall prey to these simplistic tactics, but it'd be folly to presume them impervious. Does it still happen?"
Yes, it still happens a lot... there's widespread knowledge of so-called "google bombing".. Google pops up some of its search results based on the content between an A HREF tag, as you can read about here: Google Time Bomb...
Much like security, I think this is the kind of thing that hackers and tinkerers will always find a way to exploit. The question is who can stay ahead in the race?
http://www.babysmasher.com
http://www.openingbands.com
Well this is less so when one accounts for Google's limitations. The biggest of these, in my experience (as someone who works for a site whose google rank directly affects sales) is the fact that Google apparently rarely indexes URLs that contain 3 or more CGI parameters after the "?" character.
e =4 to site.com/product/2/4/something.html, and lo and behold, the next time googlebot came by, those pages were indexed (I had verified that the problem was not that the pages had a low pagerank, but that they were not even being spidered at all).
For example, a search on google for "plaid socks" yields only 1 or 2 sites out of 100 that have 3 or more CGI parameters, when I'm sure there are many sites using very complicated urls (with session IDs, etc). Sure, this is just anecdotal evidence, but as someone whose product catalog was listed by urls that had at least 3 CGI parameters (and sometimes 5 or 6 depending on the referring URL) I can say with 90% confidence that having a "complicated" URL severely hurt us. What I ended up doing recently was using mod_rewrite to change all the listed URLs on our site from site.com/product.cgi?sku=something§ion=2&styl
What does this have to do with Google's relevance? Sure, they are returning relevant results when you search, but if they are arbitrarily not listing a site because its URL structure is too "complex" then there's a ton of possibly relevant content that they're missing. If you're someone who sells plaid socks for $10 less than your nearest competitor but Google isn't indexing your plaid socks page because of URL structure (exactly what was happening to us, except not for plaid socks) then you're really not getting the most relevant results. Which is not to say that what you DO see isn't relevant, it's just that there's possibly MORE relevant stuff that you won't ever see.
Fortunately Google has something in the works to cover this particular situation, but it doesn't really have anything to do with fixing their URL complexity policy.
rooooar
Yahoo does not charge for submission, but you'll likely never make it into their db either, because everyone submits. If you pay them $200 then you're guaranteed that they will review your site within 2 weeks, though this does not guarantee you'll be in their directory.
It's also worthwhile to mention that Yahoo's not really a search engine in the sense of something that crawls the internet looking for info; they generally rely on submissions, with which they're surely inundated, and that tiny subset of the internet is what they search.
As for sponsored links, 75% of the "sponsored links" on search engines are culled from Overture (formerly goto.com). Goto took a lot of heat back in the day for selling search results, but they've found a market in selling these results to other engines. Until like 3 or 4 months ago, their results were on Yahoo, AOL, Netscape, Altavista, and most other search engines. Then Google got into the bid-for-keywords market with their Adwords Select program. Now in addition to searches on google.com, Google's adwords show up on searches on AOL, Earthlink, and a few others. The process is basically as you described - bidding for keywords. Usually it's not worth bothering unless you're in the top 3 for that keyword on Overture, as those are the ones that show up on Yahoo (I think #4 and #5 show up at the bottom of the page). On Google I've seen up to 8 ads for a given keyword (e.g. computers) but AOL only takes the top 3 for its "sponsored matches" as well.
On Google it's important to note that the sponsored sites and the real search results are completely separate (dependent on how much you trust google, of course, but they have a lot of karma built up), and google's results are gleaned from having their robot (Googlebot) crawl the web, not from submissions; and the algorithm that ranks sites is another matter entirely. E.g. a search for "ass grabbing computers" predictably has 0 results, but there are plenty of ads for the word 'computer' that pop up.
It's doubly important to note the above about google since many Yahoo searches fall through to google when there aren't any results in yahoo's (IMO Lame) directory, so the results from yahoo are not as paid-for as you seem to imply.
rooooar
Just a shameless plug here for the Open Directory Project. Leaving aside occasional occurances of editor-fraud or editor-abuse (which are quickly tracked down by the meta-editors), this is the best way to determine a site's real value.
A human looking at the page to subjectively/objectively determine its value is something that can't be replaced by a spider and an AI program.
URL cloaking, hidden text, keyword tricks, etc... don't matter. =)
-jc
Hire a Linux system administrator, systems engineer,