Google Prefetching for Mozilla Browsers

kv9 writes "A post on GoogleBlog reveals that Google has enabled results prefetching for Mozilla based browsers, which means that the top results of queries are being loaded in the background and pages will load faster. More info on the Mozilla Prefetching FAQ and the Google Webmaster FAQ"

Comments

Watch for this...

[grub]

I can see employees being confronted for browsing pages they never actually looked at. An obvious example: innocently searching for info on the silly Vin Diesel movie "XXX" turns up a nice mix of Vin and pr0n in the top results. Presumably a mix of both are loading up in the background

Re:Watch for this...

[the_mad_poster]

Prefetching is one of those things that seems like a really great idea on paper, but doesn't hold up so well in practice.

The problem is that you have things like 'rel=next' that expect the user to go to some next "logical" page, but no structure to a site to encourage that logic. You get people upping bandwidth costs and slowing down browsing time because the site maintainer THINKS they'll go to some next page but the site design actually ENCOURAGES them to go to some other, unrelated page.

In OSS, a lot of the maintainers and coders are just "hackers" or college kids contributing bits and pieces of less broad knowledge over a bigger project team, not real software engineers who have been trained to really think through the consequences of certain design decisions. They don't research their target audience to realize that, while the prefetching feature would be great in a perfect world, the world isn't perfect, and it can really cause problems when so many people are moving from professional browsers to more amateur ones that test out these features in what they THINK is a mainly geek-oriented audience.

I could see why someone at Google might think this is a good idea, but I'd expect that a company with the means to do the necessary research wouldn't go about implementing this kind of hackish "feature set" until it had thought things through a little better.

Re:Watch for this...

[Juvenall]

To be fair though, it's never appropriate to search for Vin Diesel.

Re:Watch for this...

[argent]

You get people upping bandwidth costs and slowing down browsing time because the site maintainer THINKS they'll go to some next page but the site design actually ENCOURAGES them to go to some other, unrelated page.

There are extensive studies from third parties on what people look at and do when they search on google. And you know what, they found people tend to look at and go to the top result, and don't even glance below the top few results most of the time.

I'd expect that a company with the means to do the necessary research wouldn't go about implementing this kind of hackish "feature set" until it had thought things through a little better.

I'd expect that Google has better figures on where people go to from Google's search pages than anyone else.

Re:Watch for this...

[bfields]

Prefetching is one of those things that seems like a really great idea on paper, but doesn't hold up so well in practice.

The page you cite in support appears to be an argument specifically against prefetching pages with the rel=next attribute. As you say:

The problem is that you have things like 'rel=next' that expect the user to go to some next "logical" page, but no structure to a site to encourage that logic.

That's a flaw in firefox's prefetching logic, not in site-designers' use of rel=next, which was never intended to be used to indicate links the user was most likely to follow.

In any case, google is actually using rel="prefetch", which *is* intended for that purpose. And google's use looks pretty sensible: "This tag is only inserted when it is likely that the user will click on the first link." From experimenting it appears that it's only used on some searches; e.g. the example they give is the first hit on a search for "stanford". So presumably they have fairly good evidence that a user is actually likely to click on such a link--I suspect they have enough data on this that they don't need to just guess.

In OSS, a lot of the maintainers and coders are just "hackers" or college kids contributing bits and pieces of less broad knowledge over a bigger project team, not real software engineers who have been trained to really think through the consequences of certain design decisions.... I could see why someone at Google might think this is a good idea, but I'd expect that a company with the means to do the necessary research wouldn't go about implementing this kind of hackish "feature set" until it had thought things through a little better.

I think you're making some huge generalizations here based on very little evidence.

--Bruce Fields

Re:Watch for this...

[shaitand]

In commercial development, a lot of the maintainers and coders are just "hacks" or college grads, who know how to write a resume and interview well, contributing bits and pieces of less broad knowledge over a smaller project team, not real experienced software engineers doing what they want to do and who have the brains or inclination to really think through the consequences of all design decisions.

"and it can really cause problems when so many people are moving from professional browsers to more amateur ones that test out these features in what they THINK is a mainly geek-oriented audience."

Precisely what browsers are you referring to? Perhaps you would care to let us know which browsers your highness believes to be "Professional" and which he believes are "more amateur". In general, I would contend the latter are actually superior browsers and that is why people move to them. Every browser I know of goes through a development, alpha, and beta stage to test features before final release. Also, google is implementing this, not a browser.

"I could see why someone at Google might think this is a good idea, but I'd expect that a company with the means to do the necessary research wouldn't go about implementing this kind of hackish "feature set" until it had thought things through a little better."

Perhaps they do not use rel=next attributes and believe they have a bit more data at their fingertips than you do. Maybe, just maybe, you are the one who has not performed any research and google has in fact examined a great deal of data. Maybe that data even tells them that the number of people who continue to the top search results is staggering.

Re:Watch for this...

[anethema]

So turn it off in about:config, nothing lost.

Re:Watch for this...

[smallpaul]

Your post is confusing. First you say that prefetching doesn't work that well in practice and present a link to Simon Willison's blog. But the blog says that prefetching is an "excellent feature" except for a couple of quirks in Mozilla's implementation. Google does not trigger those quirks so they are irrelevant.

Then you go off on a tangent about how "real software engineers" think through their design decisions more than "open source hackers". This is totally contrary to my experience. I would more highly rate the software engineering of Mozilla against Internet Explorer, Unix versus Windows or Apache versus IIS. I could go through a long list of brutal design decisions in commercial software that did not hold up in the real world but I'll just mention Clippy and the Windows registry as two high-profile examples.

Finally, it is Google, a commercial software services company that is the topic of the article. So your whole argument is self-defeating. Either Google doesn't conform to your vision of real software engineering or the feature is not really at odds with real software engineering.

Re:Watch for this...

[switcha]

Hi there. I just came across your post. I was searching for "Vin Diesel" on Google...

Re:Watch for this...

[damiam]

Google does use a redirect script, but only for a small percentage of sessions. And you probably wouldn't notice; they use Javascript to display the original URL in the status bar. (IIRC)

MSIE/MSN

[mirko]

Does somebody knows whether MSIE and MSN collaborate the same way?
Anyway it could be obvious that Google tries to establishes such alliances against his main concurrent (besides Yahoo).

Padding?

[Quixote]

Would prefetching pad the click count for the ads that Google shows along the side? I know, the client (Moz) adds a
X-moz: prefetch
header, but how many server admins log this?

Re:Padding?

[nautical9]

If you're referring to the sites that google's adding the prefetch links for, then no - since there's no "clicking" happening for anything but the result page. If anything, the impression to click ratios will go down, because the page (and the google image ads on it) are being loaded, but the client will never click on it because he never saw it.

If you're referring to potential click-fraud that a malicious site may do by adding a bunch of "prefetch" links on their page that points to the ad, I seriously doubt it, since no one but the client's machine knows what the links are (since I believe they're loaded at the time the image is generated).

Perhaps an enterprising fraudster may write some clever javascript that waits for the google ad to load, and then generates the prelinks - but I doubt the browser would then notice the change and go prefetch them. Besides, it'd be easier to just make an invisible frame and set it's href location. But again, I doubt you can dynamically figure out what the google ad click URLs are with javascript alone.

Link is broken

[KinkifyTheNation]

Replace the %23 with a # and the url will work.

how about a link that works...

[... James ...]

Slashdot kills the # character in the URL: prefetching faq

This is Potentially Dangerous

[cyranoVR]

Ever heard of the concept "one click and you're guilty?" Users of this feature who unknowingly perform a search that returns results containting offensive/illegal content may find themselves being prosecuted by local, state or Federal authorities...

Proof of concept: Google caught in anti-Semitism flap. Replace "anti-semitism" with "child pornography" and you'll understand what I'm getting at...

Re:This is Potentially Dangerous

[Rakshasa Taisab]

Note to self; Remember to turn of network.prefetch-next when googling for "child pornography".

broken link, here is the correct one

[grandmofftarkin]

a href="http://www.google.com/help/features.html#pre fetch"

Re:Links

[sharkey]

BUGzilla blocks Slashdot referrers, not MOzilla.

Re:I never thought speed was a problem with Google

[TobascoKid]

It's not the speed of google - it's the speed of jumping off google and on to the website "most likely" to be what you're searching for.

Ack!

[Anonymous Coward]

I'm using a modem you insensitive clod!

Yes - from the page with the bad link

[slash-tard]

1. Type "about:config" the address bar.

2. Scroll down to the setting "network.prefetch-next" and set the value to "False".

Re:Yes - from the page with the bad link

[anethema]

It is better to put this in a user.js along with all the other settings you want to keep between installations.

Comments

[hendridm]

3. I want to block/ignore prefetch requests. What should I do?

To block or ignore prefetch requests (from Google and other web sites), you should configure your web server to return a 404 HTTP response code for requests that contain the "X-moz: prefetch" header.

Sucks for those of us on shared providers, I guess, who don't want this so our bandwidth costs don't increase.

I wish they had an option in the Google preferences to disable this, as I don't need a slower connection. Fortunately, you can disable it:

Yes, there is a hidden preference that you can set to disable link prefetching. Add this line to your prefs.js file located in your Mozilla profile directory:

user_pref("network.prefetch-next", false);

It would be nice if there was an option in Firefox prefs to do this so I don't have to remember it every time I reload.

Re:Comments

[justforaday]

It would be nice if there was an option in Firefox prefs to do this so I don't have to remember it every time I reload.

Doesn't changing this value on the about:config screen do that?

Result, not results, and then still not always

[Anonymous Coward]

Only when google is confident that the top result is the one you want - the one link that the vast majority of people actually click - do they include the prefetch link for that one resource. Go and try it for yourself, and look for prefetch in the source. For the vast majority of searches, it isn't there. Only when looking for the authoritive resource (such as stanford.edu for "stanford") is the prefetch link actually there.

Sure, their metrics might be off at times, but the way this has been implemented is definitely a good way, and will be very helpful for users of all browsers implementing prefetching (which currently is gecko-based only afaik, but could easily enough include opera and safari and such as well in the near future).

How to turn it off yourself

[dave7e9q]

Type about:config ... then scroll down to network.prefetch-next ... double click it to "false" ... all done.

This is great!

[Slashcrap]

Unless :

You have an ADSL line with a really stingy cap (for instance BT in the UK offer a cheap service with a 1GB/month cap). I'm sure their customers will be happy about downloading pages they won't read.

You're a web admin that pays a lot for bandwith. I bet they'll be really happy that lots of people will be downloading their pages without ever looking at them.

You're at work surfing through a proxy that does filtering / logging and there are some dubious sites that get pre-fetched for you. Enjoy getting sacked for something you didn't do!

Well, I don't know about you, but I'm struggling to see any drawbacks to this great new technology!

Re:They turn it off for you.

[otisaardvark]

Um, if you read the Mozilla Prefetch FAQ, you'll learn that this only happens when you are not using bandwidth for something else initiated by the current Mozilla application. Worry :-)

Trouble at work, trouble with law

[rhh]

Not only could this get you in trouble by inadvertently downloading porn at work, but you could download even more incriminating things.

Say for example you were searching for info on that convicted sex offender that moved into your neighborhood or searching for news on terrorist attacks. Prefetching could potentially have your computer downloading things you wouldn't otherwise download and that could get you in real trouble.

Re:Trouble at work, trouble with law

[Rommel]

This is just another reason the current practice of criminalizing the possession of knowledge is crazy.

It should be off by default....

[jonr]

I think it would make more sense if network.prefetch-next would be set to false by default. Then gearheads could turn it on if they wanted.

Re:But usually the top links on google

[md27]

You do have this chance to vote, if you find a useful page clicking the "More like this" link, not only gives you pages tuned closer to that page, but it also tells Google that's what you wanted to see when you searched in the first place.

Even More Problems

[Valdrax]

Forget the Feds, you're much more likely to get nailed by your IT department for this. I wonder if a user who was unaware of this feature and got fired thanks to links loaded by it could sue the Mozilla Foundation. I can just see some malicious little asshole putting hidden (via color) links in their webpages that download utterly offensive crap just to see if they can get someone fired. I especially expect this sort of thing from the same sort of Slashdot trolls who posted that infinite pop-up of gay porn thing in the Firefox Hacks story.

I also expect that this will be abused by unscrupulous websites who want to run up their ad revenue by having people preload a page full of ads. Many people have already expressed concerns for those who have slow connections or who do not have unlimited access. This could also be used by spammers to verify people who are smart enough to have web-bugs disabled via cookie and image blocking on emails but who don't know about preloading if the Thunderbird people enable this in email (which would be foolish beyond belief).

I just think this concept is a horrible idea.

google tracks clicks sometimes

[slashkitty]

They do put in click tracking links sometimes.

For example if you search for: ~hot

You'll get the tracking links. I think it's random on many searches, but on ~ searches, they always have it.

Re:google tracks clicks sometimes

[red tiger]

Actually the tilde-feature is properly documented.

It is called synonym search and it really does some dictinary lookup, as it doesn't search for just the term entered but also its synonyms.

Seems quite useful to me... To be honest, I had never heard of it before I read this thread here :)

Re:Nice

[KFury]

Now every 56ker is going to move away from google.

Why?

Lie the article says, Moz/FFX only uses bandwidth you're not already using, so it won't make any other operation slower, and if you're on a slow connection then prefetching a page saves you even more time than if you're on a fast one. What's the use case that would have you moving to the door?

Re:PS...

[elgaard]

Isn't it simpler to set your google preferences to do that?

http://www.google.ca/preferences?hl=en

Another concern

[DaveJay]

Other people here have already discussed the "you'll get in trouble from work/authorities for prefetching things on to your computer you don't even know you're loading" deal, so I won't touch that. They've also discussed the "You'll use more bandwidth" thing.

Here's my complaint, from an entirely different direction: two years from now, is every default installation of Mozilla and/or Firefox going to require me to change a laundry list of preferences in order to avoid features I don't want?

I mean, go ahead and put these features in, but don't activate them automatically: do what Opera does (asks if the user wants to activate a feature) or just leave them off by default, and add a menu option to turn it on.

Having these things turned on by default is going to be an inconvenience going forward, and smacks a bit of elitist "we know what's better for your web browsing experience than you do" attitude, you know what I mean?

At this point, I'd be thrilled with setting optional parameters like this to 'off' by default, and updating the default installation home page (visible on first execution of the app) to a page listing "Great optional features", along with buttons to turn them on and a quick note on how to turn them back off if desired.

Vulnerabilities and prefetching?

[Chris Kamel]

So what happens if you prefetch a page that stresses some vulnerability in the browser? Does it get to run even though you didn't really even browse to the page....