New Google Search Index 50% Fresher With Caffeine

Ponca City, We love you writes "When Google started, it would only update its index every four months. Then, around 2000, it started indexing every month in a process called the 'Google dance' that took a week to 10 days and would provide different results when searching for the same term from different Google data centers. Now PC World reports that Google has introduced a new web indexing system called Caffeine, which delivers results that are closer to 'live' by analyzing the web in small portions and updating the index on a continuous basis. 'Caffeine lets us index web pages on an enormous scale,' writes Carrie Grimes on the official Google Blog. 'Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day.' Now not only does Caffeine provide results that are 50% fresher than Google's last index, adds Grimes, but the new search index provides a robust foundation that will make it possible for Google to build a faster and more comprehensive search engine that scales with the growth of information online."

Comments

Re:It's called the metric system. Use it.

[Anonymous Coward]

Or maybe is it a pebibyte ? Who knows ? http://en.wikipedia.org/wiki/Pebibyte

Re:It's called the metric system. Use it.

[the bluebrain]

Caffeine takes up nearly 100 million gigabytes of storage in one database

A million gigabytes is what we call a petabyte.

And by "we" you mean "us who don't mind being off by a factor of 100 or so"?

Re:And yet Google adds less and less to my ....

[Mana Mana]

: Plus when Google (and Yahoo) killed off Usenet (with an assist from Andrew Cuomo) the utility of
: the Usenet information structure has been destroyed (which the world is still trying to recreate
: with Keywords).

What are you talking about? What has nearly killed Netnews is the elitist ass, pedant, recalcitrant ways of news oldtimers and wannedtobe ol'timerz. Back in the late 1990s UCE, UBE, SPAM! was getting seriously annoying but systems like Moose that allayed or tried to eliminate it were just not adopted by the powers that be 'cause you know the shit had to be studied, couldn't rush it, you know "this is Usenet, man." Fucking Google had nothing to do with it, they didn't steal anything away* when they acquired DejaNews from its failed self. Their UI was so-so when it was Mutt like ASCII arrows for thread structure, then they improved the shit out of usefulness, I thought.

Usenet was/is? great for the warez, the p0rnz, the lulz but the level of shit to gold went absurd with the torrent of SPAM--thanks of the endless debating server operator Order.

* I hate to be teaching you in a crowd like this, but if you could---"poof"---duplicate my BMW M3 just like that what the fuck would I care. I would still have my M3. "But, but, but." Butt your wife's sweet ass, I'd duplicate and ride her ass too. What goes around comes around and we'd all be better for it, if the **AA suits understood that the cocaine and hookers for all would be on them.

Re:Altavista

[IgnoramusMaximus]

As I said, you are completely out of touch with reality if you think that a majority of users have any interest in things like that. Making the claim that management somehow snuck an almost-universally hated feature in is absurd and only harms your credibility. You do not represent the majority of users. Slashdot does not represent the majority of users.

Right ... and all the people begging me (and any other IT professional I work with) for help in turning these "features" off also do not represent the majority of users. In fact the majority of users I managed to meet in my 30 and some year career apparently does not represent the "majority of users" .... only you do. Congratulations. Probably that is because of your self-awarded "credibility". I fully expect that you were in the "majority of users" who positively loved "Clippy" and had him turned on permanently.

If you don't, then why are you arguing? I'm not arguing that fade-in is worth it; only that the majority of users wouldn't care and/or be aware enough to opt-out.

There is a major difference between "not caring" and not knowing what to do about it. Thus what usually happens is that the issue becomes a low-key nuisance about which users cannot do anything or about which they are afraid to ask (because it seems like they are somehow supposed to adjust to it and they do not want to appear stupid or "behind the times") and after a while they become so accustomed to it that they learn to tune it out, like rheumatics learn to live with their pain. Only when reminded, usually during some unrelated conversation with IT personnel they suddenly exclaim: "and would someone do something about this fucking xyz thing that drives me crazy for months now!!!". That is the standard pattern which I have witnessed countless times.

Google likely included it because of the positive reaction users had toward Bing's related searches in their sidebar

Correction: they included it because some bozo at Google panicked about Bing doing something that they did not have, regardless of any advantage, usefulness or other factors. This is a standard pattern of behaviour of incompetent management who is faced with a possibility of somewhat viable competition. They simply attempt to emulate all and any features of the competitors in hopes that a "shotgun" approach of "and we have that feature of theirs too!" will save their market position .... somehow.

A situation all the more amusing given that Google already had an equivalent (and superior) functionality resulting from including "image", "shopping" and the like sample results as a section inline, which required no Ajax functionality.

At the recent WWW2010 conference I attended, the side bar was viewed positively by engineers from Bing, Yahoo, and Google alike. Considering they have access to real-world usage data and you do not, I'm inclined to take their side. In fact, Bing claimed the UI redesign _alone_ significantly increased traffic before any backend changes had taken place

Oh for fuck's sake, I have the real world usage data in form of thousands of corporate users, probably far more scientific then theirs since I get to see the way people at work actually interact with Google, something no server logs at Google can ever show. And listening to claims of companies spouted at, of all places, a conference, where every one company tries to out-posture all the others! That's like going to a Comdex circa 1994 and hearing the ever-present IBM salesmen and engineers positively promising you that OS/2 will take over the world! Guaranteed! Or going to some programming conference and learning that Smaltalk is all the rage that will replace every other language and paradigm ever invented! Etc and so on.

There has to be some kind of special category of gullibility just for the "serious" (as opposed to the get-away-and-party kind) trade conference goers.