Reports Of Google's Demise Exaggerated

Several readers have written in to tell us about this GeekPress story in which the popular and effective search engine Google is scammed by an adult site operator. Interesting story. Too bad it isn't true. "My basic conclusion," says Google's CTO, "is that the scam" -- not that I think it is a scam -- "didn't really 'work.'"

Google's advantage, of course, is that its page-ranking means that the Web sites you find are less likely to be affected by how many keywords they can cram in, and more by which other important sites are linking to them. The theory is that lame sites won't be linked-to by important sites, and that therefore they won't show up high in your search results. And the theory usually works pretty well, which is why Google is my preferred engine.

The GeekPress article says that a search on "Liv Tyler nude" has as its top results some links all going to the same site which has Liv Tyler (allegedly) nude. Well, OK then. If you think that's a problem.

Google's CTO Craig Silverstein comments that that particular search query doesn't, "as far as we can tell, have any good results -- in our spot check, for instance, we couldn't actually find any Web sites that show Liv Tyler in the nude. When there are no good results out there, Google's results can be somewhat arbitrary, so it's not particularly surprising this site was first."

When I randomly checked the names of more popular actresses, plus the word "nude," the supposed-scam in question didn't pull down any especially good hits.

This was confirmed by the adult site itself. When I e-mailed its representative, he claimed their ranking for more popular celebrities like Cindy Crawford and Pamela Anderson were way down the Google list: 22nd, 38th, and worse.

And, protesting the "scam" label, he pointed me to a good article on bridge pages. The technique they're using is a popular method of getting hits which -- as long as the destination pages bear relevance to the search terms, which they here do -- is in the gray area usually considered aggressive self-promotion. It's a trick more or less ignored by the search engines until it's combined with other less-savory tricks.

(That article, and most of searchenginewatch.com, makes for fascinating reading if you're interested in the arms race for your eyeballs being fought between engines and webmasters.)

Also, the adult site operator says his site has gotten only 400 hits a day from all the bridge pages they've set up. It's hard to argue that just a few hundred clicks over to Jane Doe nude represent an extraordinary hijacking of the search term "Jane Doe nude."

Google does refine their algorithm, which incidentally like all search engines' is kept secret to avoid giving Web-spammers an edge. You may remember last year's joke of the search "more evil than Satan" pointing (mistakenly, of course) to Microsoft's homepage. As their founders comment in the recent MIT Technology Review interview, this was a little embarrassing for them, and the engine was tweaked to fix it.

And Google's CTO isn't ruling out more tweaking in the future:

In any case, we know our scoring scheme isn't perfect -- even when the sites in question aren't trying to fool us -- and we're always working to improve it. Often the problem isn't, "Why did this bad site score high?" but rather, "Why did these other good sites score low?"

We're always looking at queries that give strange-looking results to get a better understanding of how our scoring can be improved. Whether the "xnude" queries will result in tweaks to our scoring, I can't say, but we'll certainly be adding them to the test cases we look at.

Short version: the arms race continues; Google still kicks butt.

Comments

Funniest quote...

[Ami Ganguli]

Whether the "x nude" queries will result in tweaks to our scoring, I can't say, but we'll certainly be adding them to the test cases we look at.

Yeah, I bet you will!

Ok, let me this straight. Google is going to employ people to 1)look for porn on the Internet, 2)decide which is the best porn, and then 3)improve their search engine to rank it properly.

I'm sure I can find quite a few volunteers in my workplace who are already doing 1 & 2. I bet they'd love to have this job.

So...

[American AC in Paris]

To paraphrase:

Somebody wrote about a scam on Google, but it turned out to be completely ineffectual and nothing of any real importance.
Liv Tyler nude.
Just thought you'd like to know.
Liv Tyler nude.

Ayup. Stuff that matters.

This is not acceptable!

[Duxup]

I think this needs to be checked into further. I would personally like to offer my services to Google to verify the accuracy of all possible searches regarding popular actresses, the word "nude", or related terms.

Google if your listening, feel free to contact me, I can send a resume and get to work immediately. I can also send your a detailed report regarding several popular actresses and some research I've already began working on in my spare time.

Before we get the trolls ranting about search.....

[Xenex]

engines - Have you TRIED Google?

Before Google, I'd given up on search engines. To but it bluntly, the things were shitful. Rarly would a search constantly find main useful sites, and stuff totally offtopic was commonplace.

Then I heard on Geeks in Space Rob and co rant about Google coming out of beta. I mustn't have read /. enough around then, but I went to this 'Google', and what I fould was the holy grail of search engines.... or something.

Google CONSTANTLY gets great hits on resonable searches (resonable = not looking for porn and pathetic crap like that). Not only that, but Google loaded INSTANTLY.

Google brought me back to search engines, and made the web useful again. Google REALLY s a step above it's search engine competition. If you are one of those people that have traditionally always used one engine, give Google a shot - I swear you won't be disappointed.

The good folks at Google don't have any serach problems when I try, and I think this is a one-off. And considering the person was just looking for nude porn pics of some girl that there is probably no nude pics of (I don't know who the girl is... maybe there are), then Google can hardly be blaimed for crappy results - of course it can't find something that doesn't exist ;)

Anyway, enough Google ranting for now. Just remember, the fisrt time you use it, you'll know you are on to a good thing....

(Look at me, you'd think I was advertising... ;)

A better way?

[deefer]

The last sentence says it all...
"Otherwise we won't be able to find anything on the web"
I know the gags about all web searchs returning at least one pr0n link, and they are exaggerated to a certain extent. There is some truth in it, though.
But the lack of honesty on the internet appalls me. As an old schooler (I remember gopher, before this new fangled www thingy), I'm used to being able to find things quickly, and only find things that are relevant.
With the proliferation of personal home pages, unscrupulous company webpages exporting lying meta tags, I'm disappointed in the web as it has turned out.
Pr0n sites are they best and worst thing on the internet. Best, not for the reasons you'd think, but because the pr0n sites drove the need for high quality image and moving picture compression, and provided the need for higher bandwidth availability. But now, they are ruining our internet, along with the spammers. That said, AllTheWeb does an excellent job at stopping "adult" links being returned from a search, and it's bastard quick, too.
Why not _force_ all pr0n sites to have a .xxx TLD? Easily spotted by search engines, web filters and all. So all of us who can handle the internet in it's glorious tangle can find what we want, and avoid what we don't. And segregate other sites into more stringent TLD's - like .tec for tech sites, .sel for ecommerce, .hmr for funny sites etc etc etc. Everyone benefits that has a legitimate site, effectively killing these ad-laden dross sites that popup 10,000 more windows when you try and close them. The only problem is policing this effectively - maybe have an optional membership to a regulatory agency, which allows you to publish a "certified" tag, which can be a criterion for web search engines. The only problem again, is how this agency supports itself financially.
What do you guys think?

Strong data typing is for those with weak minds.

Re:A better way?

[Mike Connell]

> Why not _force_ all pr0n sites to have a .xxx TLD?

Same reason as ever. Who gets to decide what's a porn site, and must go into .xxx?

Take for example these categories:

www.tightskirtspage.com, appears to be dead (might have been a while). It used to be a free site for people that had a thing about tight skirts. Porn?

Feet. Lots of people into feet. Porn?

Lingerie/Swimwear sites. Porn? Are shops that sell this stuff not porn?

Actress of the month-fan-site. Porn? What if they'd done some porn in their "previous career" prior to getting to be a famous actress. Those shots will always appear on fan sites. Does the fan site then have to move to .xxx?

And last, but not least: Who gets to chose? Some wacky Christian/Whatever fundie? Some drooling pervert?

Sure, it's a nice idea. I'm all for honesty in web pages, death to pop-up windows, and dont lie in your META tags, but you can't enforce it.

0.02,
Mike.

Re:A better way?

[Vryl]

Man, I use googles 'safe search' to cut out the pr0n and limit my queries to english only pages.

It works bloody bagus maaate.

Basically, google rocks, especially on older computers, or in lynx.

In the old days, hotbot and altavista were great, but became filled with crud and altavista's 10 result limit peeved me.

When google came along it was a breath of fresh air, and the features it continues to add continue to add value, not subtract like the mess on other engines.

Raging is pretty good, but I don't see the point, since it is a shameless clone of google.

Re:The fun you could have....

[deefer]

How you could manipulate queries and influence so many minds

They already do.
An ad mans wet dream

Not long before companies can pay to have their products boosted to the top of the search results list...

Strong data typing is for those with weak minds.

Well actually...

[DragonC]

I do run my own search engine.

I run websearches.net which is a topic based search engine. I use my own ranking system, but the great thing is you cannot spam it at all.

Why? Well firstly to be considered is must pass the theme check, then it checks for page spam (duplicated keywords, repeated patterns etc.) then it checks to see what level sites it links to and from.

You still get people searching for strange terms (you can never get away from those!!), but at least I have the luxury of laughing at how people use the system. Such as the amount of people who search for "Linux" on the Linux search engine! It's meaningless. It must be a linux site to get in the database in the first place.

Dragon.

It's something we do!

[horza]

I worked on a commercial web browser, and users would tell us their favourite porn sites weren't working. So we would have to look at them (for scientific purposes, stop giggling down the back) and tweak the rendering code to be a bit more tolerant of their often sloppy HTML or whatever trick they were trying to pull. Porn sites have always been way ahead of the curve in terms of Internet technology so if a browser will cope with them it will cope with most sites. We did find it highly amusing to be handed very formal looking project management sheets, which were then splattered with references to sites like 'sex.com'.

Phillip.

More Evil Than Satan Himself ...

[Vryl]

... *used* to go to the microsoft site. Now it just returns stories on how if you enter 'more evil than satan himself' into google, you get the microsoft homepage.

This is fascinating, because the story has invalidated itself.

Some weirdarse self-referential hoffstadteriam net.paradox has gone down.

I assume the next step is that stories on how 'more evil than satan himself' *used* to refer to the microsoft homepage get top billing, and so on ad infinitum ...

... or maybe not ...

Google's underlying problem is real.

[dash2]

Whether or not this scam worked, the problem remains that Google's algorithm works by taking web pages as authorities on other web pages (there's a good explanation of how this stuff works at Quiver's website, in their technical white paper.) This means that if enough web pages can get together, they form a cosy little club, each certifying the others' authority. The problem is not so much that this can be "scammed" - although I see nothing to prevent this - as that it systematically biasses the results in favour of large websites or conglomerates of websites (cf. Internet.com).

Ultimately the best way of finding good sites has to be by getting humans to find them, but without requiring the huge expenditure of effort that actual directories need - even the Open Directory is nowhere near a full reference. This is what "favorites directories" like Blink and Backflip are about, as well as Alexa which uses information based on people's surfing patterns. Maybe an open source implementation of some of these, built into Mozilla, would be a good idea.

Re:I am about to give up Google

[StrontiumDog]

Try searching for the string "a-life". Google doesn't support this. Using altavista you can at least get some results.

But searching for "Alife" on Google works like a charm. Google is a tool. Judicious use of keywords and alternate keywords makes it a very powerful one. Other search engines pale in comparison.

Re:The fun you could have....

[DrWiggy]

"Vandalising" is not quite the correct term, and it's a bit too subjective. One man's vandalism (graffiti) is another man's art. The problem is that there are a conflicting set of values in force here, just as there are in real life. Whilst absolutly despising commercialism in the "ads everywhere you look" sense, I understand that it underpins the economics of our society which allows me to live in a nice 2-bed city centre aparment, eat good food, have 2Mb/sec ADSL into my home (which is still rare in the UK), and in fact almost every technological advance made in the last 50 years at least has been due to commercialism.

In addition, without porn, warez, hacking, etc. we wouldn't have some of the most useful technologies around on the net for what we do in our day-to-day lives. Porn gave us streaming live video technologies, driven forward the requirements for better video and image compression, transparent cache engines, etc. whilst "warez" in all it's forms gave us not only the idea of "free" software, but DivX movie compression, requirements for bandwidth shaping in ftp servers, etc. and hacking gave us the motive to invent and make a standard protocols such as SSL, S/key, OPIE, etc. all of which have provided the basis for legislation (in the UK at least) for acceptance of digital signatures, etc.

It has often been said that the guys coming up with the porn sites are on the "bleeding edge" of technologies. This is because the market is so competitive, they are looking for new ways to get an edge over the next guy. It might have been streaming video, new ways to attract customer loyalty, those annoying windows that appear when you close a browser on a porn site, whatever. The point is, it didn't come about because it was porn, but because it's the most competitive industry currently on the net.

Perhaps we can make do without it all, but without highly competitive market places on the net, technological development will starve. This would be bad for all of us. I don't think even the Amazon vs. everybody else market is competitive enough to really push things along just yet.

To finish off, I'll also just remind you about the success of VHS and digital cameras. The reason why video recorders/players in the home along with video cameras were so successful was porn. Think what you like about people wanting to enjoy films at their leisure at home, but porn is what sold them. The ability to watch porn at home, at relatively low cost, was the key selling point. Digital cameras are the same. People want to be able to take saucy photos, but don't want them to be sent off for processing. Porn drove the market. I'm not saying it's right, or that it's wrong, but the only reason why you have a cheap webcam on your desk is because 3 million other people want to take photos of their girlfriend in stockings, in the same way that I have reasonably cheap bandwidth to my home is because half of the country wants to be able to download porn quicker. Remember that.

This is redundant, I know, but if you could tell me where that village is, I'd be interested - I refuse to believe that such a village still exists right now, just 150 miles from where I live. If it is there, I know where I'm moving next. :-)

If you really want pron

[Anonymous Coward]

just go HERE

Trust me, it isn't goatshit or whatever

Wouldn't work anyway

[Stephen]

I think the scammers hadn't understood Google's algorithm. The way I understand it, creating a ring of pages doesn't work. You only get a high rank if other highly ranked pages point to you. So if you create a ring, none of your pages get high ranks, unless other good people point to one of them.

From Google's page:

Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important."

Or from a recent interview with the inventor:

Q: Couldn't I create a series of Web pages at every site offering free space, link them to each other, and thereby create higher rankings for myself?
A: No, that wouldn't work. The way the math is designed, you need external approval.

Response: Still Scamming Google

[geekpress]

I think that the folks at Slashdot missed the point of my article. And from what I have heard and read, the folks at Google don't quite get the problem. So I wrote up a response.

In particuar, it addresses two claims:

1. There are no good results to a search for Liv Tyler nude. (Google)

2. The scam doesn't yield good Google results on popular actresses. (Slashdot)

You can find the response, titled Still Scamming Google at GeekPress. For the sake of my Slashdotted server, I'm reprinting it below:

***

Still Scamming Google

The Case of Liv Tyler

My article on Scamming Google has some unexpected results. The most interesting is that the folks over at Google deny that the scam has any significant impact on the accuracy and usefulness of Google search results.

Slashdot, for example, quotes Google's CTO Craig Silverstein as saying that the Liv Tyler nude search doesn't, "as far as we can tell, have any good results -- in our spot check, for instance, we couldn't actually find any Web sites that show Liv Tyler in the nude. When there are no good results out there, Google's results can be somewhat arbitrary, so it's not particularly surprising this site was first."

As much as I love Google, I must disagree. I scoured through the first 30 results of the Liv Tyler Google to prove my point. Here are the results:

Index Pages: Most of the returned links were to the index page of a porn site, such as link #3 and link #6, both of which take you to the same JavaScript popup hell site. (I didn't bother checking the rest of these pages, because they obviously didn't have Liv Tyler nude on them.) These links are obviously not good Google results.

Scam Pages: Then there were the scam links, with fake discussions of Liv Tyler nude, such as link #1, link #2, link #4, and link #7. These links are also not good Google results.

Nude Liv: Google did, however, return some pages with Liv Tyler nude on them. The best page is probably link #5, which popped up after three scam pages and one index page. It has thumbnails of various movie shots of Liv Tyler in the buff. Link #25 and link #30 also has nude pictures of Liv. (I also noted two links somewhere between scam and the real thing, such as link #11 and link #21. These pages have non-nude pictures of Liv Tyler and a ton of links leading to various celebrity nude sites.

I also randomly checked a few of the low-ranked pages, with some unexpected results. For example, link #76 has a genuine discussion (!) of Liv Tyler's nude scene in Stealing Beauty. Link #116 has a nude picture along with a filmography. Link #174 is an Batman-ish erotic story with Liv Tyler playing "The Huntress." Link #192 and link #63 have fake nudes. Link #62 is a list of various Liv Tyler pages, some with a bundle of sexy images.

So, contrary to what the Google people say, there are good results for Liv Tyler nude. Google just isn't putting them at the top of the list.

Random Results?

But Google's problems do not end with the jumble of bad results for the Liv Tyler search. Google repeatedly puts the fake discussion pages of Nude Celebrity World News at the top of the search results in a huge number of searches for nude celebrities. But don't take my word for it. Google for yourself using the list of celebrities created by Nude Celebrity World News (modified to automatically search Google). You'll see the domains of www.jennifer-smith.com, www.news-in-review.com, www.find-thys.com, www.celebrity-locator.com, and www.celebrity-fans.com at the top of the list more often than "randomly."

Personally, I checked 50 of these searches, making sure to hit popular female celebrities like Meg Ryan and Heather Locklear. Here are my results:

3 searches yielded all top five search results (e.g. Kathy Bates).

9 searches yielded all top four search results (e.g. Bo Derrek).

16 searches yielded all top three search results (e.g. Linda Hamilton).

3 searches yielded both top two results (e.g. Lara Flynn Boyle).

1 search yielded the top result.

In all of the above searches, I just counted the set of top results, ignoring the matching results lower in the top ten.

Most importantly, however, is the fact that 16 searches for popular celebrities had at least one result in the top 10 (e.g. Toni Braxton, Janet Jackson, Nicole Kidman, Meg Ryan, Heather Locklear, Alicia Silverstone, and Sandra Bullock).

Only 2 searches did not yield any results in the top ten: Demi Moore and Jennifer Lopez.

These results are not random, as Google claims. Clearly, the sham discussion pages created by Nude Celebrity World News have worked their magic on Google. The proof is in the pudding and the pudding is in the search results.

So I stand behind my the point in my original article: Google has been fooled into repeatedly returning, as highly-placed results, pages which any human can identify as search engine spam.

***

About the Author

Diana Hsieh is the owner and co-editor of GeekPress, an irreverent filter for the most unique and interesting technical news of the day. She also sporadically writes and lectures on philosophy, Objectivism in particular. She can be reached via e-mail to diana@geekpress.com.

© 2000 Diana Hsieh. Permission to reprint will be granted upon request.

-- Diana Hsieh

Re:Response: Still Scamming Google

[Jerf]

I'm sorry I don't have moderation points today or I'd mod you up :-)

Two honest questions:

Google uses some sort of metric-of-trust system that is supposed to prevent this sort of abuse. None of us know the internal mathematics of this system, so we don't know if self-referential loops are given no weight, or merely reduced weight.

Given that a search for "[celebrity] nude" has little information to go on, significantly less then "[celebrity] nude pictures", as demonstrated by your finding some interesting results like honest conversations, exactly how is Google supposed to tell that these pages are fake?

If one could see the trust metric on all of the search page hits, I'd lay money that as compared to the trust metric for Microsoft, none of those pages are very trustworthy. There are probably a couple of external links validating the "Nude Celebrity World News", and despite the self-referential loops in their pages, this slight boost may have been enough to give them the advantage over all of the other results you cite, which are meaningful only in the broad sense... I can't say any of the ones you cited are "definately" what "[celebrity] nude" would be searching for.

Thus, in all probability, we are looking at a "fraud" where on a 100 point scale that I'm making up right now, some site scams its way into giving itself an extra point. w00p! I don't think this scam could even come close to displacing the Microsoft Windows Home Page from the top result of searching for "Windows", which may have a trust of 99.9%. The only reason you can even see the effect of this "scam" is because you are plumbing around in the lowest trust areas of the Google database. In four weeks (or however long it takes Google to index these pages) I'll lay money that the top two hits for Liv Taylor nude will be this slashdot discussion and your site. The trust metrics of Slashdot and the one your site obtains by being linked from Slashdot will blow this "scam" out of the water.

Unfortunately, this discussion will also validate the Nude Celebrity World News (which is why I'm not linking to it here). In fact, this scam will work vastly better now that it's only two clicks from Slashdot. To really pull this off, you need trusted domain names, which we just handed them on a silver platter.

Looked at in this light, I think Google's reaction makes sense. I regret needing this lengthy explanation, but my questions are, what exactly do you think Google should do, and in light of the fact that the "scam" is probably almost (but not quite!) complete ineffective, why is this a problem?