Google URL Index Hits 1 Trillion

mytrip points out news that Google's index of unique URLs has reached a milestone: one trillion. Google's blog provides some more information, noting, "The first Google index in 1998 already had 26 million pages, and by 2000 the Google index reached the one billion mark. Over the last eight years, we've seen a lot of big numbers about how much content is really out there. To keep up with this volume of information, our systems have come a long way since the first set of web data Google processed to answer queries. Back then, we did everything in batches: one workstation could compute the PageRank graph on 26 million pages in a couple of hours, and that set of pages would be used as Google's index for a fixed period of time. Today, Google downloads the web continuously, collecting updated page information and re-processing the entire web-link graph several times per day."

Comments

Some numbers

[Reality Master 101]

Counts of words:

the: 18.3 billion pages
a: 23.9B
0: 12.7B
1: 25.4B
in: 17.1B
I: 10.2B

I know these numbers aren't exact, but you'd think one of them would be over 100B if Google is really indexing a trillion pages. What's on them? Anyone find any keywords that produce more?

Re:Some numbers

[Shaitan Apistos]

My Hobby

Attributing my sources: http://xkcd.com/369/

In , my humble opinion my usage of "My Hobby" was sufficient attribution, all by itself.

What's going on with the founders' studies?

[bogaboga]

This might be off-topic but I wonder what's going on with Sergey Brin and Larry Page's [PhD] education? Just wondering...did they give up?

Re:Wow, that's a lot of porn.

[sweet_petunias_full_]

"the web is something like 42% porn"

That probably stopped being the case after namespace speculators started buying up expired domains in large numbers just to put up a mildly useless index on *each* and *every* site to collect ad revenue or marketing statistics off of unwary visitors. I would also include typosquatters in that category, and maybe someone else can name a few other examples of utter namespace hogging uselessness.

Whatever it is, you can rest assured that it's mostly repetitive trash... no need to stand in awe of it.

Re:How long till..

[blahplusplus]

"I'm more interested in when Google starts returning relevant results to my queries.

I can't believe that I'm the only one that finds Google's quality of service somewhat below par."

You're not the only one, but for the most part it is better then most other search engines out there. The real problem is spammers and paid advertising, I think spammers have really made search frustrating for a lot of companies. And ad companies pay other people to promote their sites for them (digg, slashdot, etc). I've noticed the increase in spam-vertised websites in search results for a lot of things.

Personally I think the idea of sharding and search being more specific for what you're looking for is needed. I'd like to see a google with 'tags' and a delicious interface, things like educational institutions and universities get lumped into their own search engine space for instance, this would help narrow down what one is looking for, although it would take time and feedback to design something well for other areas. The fact is that search results get diluted as you put more and more stuff online (numbers and geometric scale).

For fun, I've noticed stumble upon and del.ico.us are not bad alternatives when looking for new and interesting sites without having to use search