Google Plans to Reveal Some of its Code

Andy Beal writes "According to Australia's The Age, Google plans to reveal some of the code it uses to great success. It says ' "The time has come for Google to "give something back", Wayne Rosing, the company's vice-president of engineering, told students while on a recruiting drive in Melbourne last week. "There have been a lot of conversations in the company in the past two months about (how) . . . it's time for us to give something back. So our technical director, Craig Silverstein, has started a project to look at all the Google code and start figuring out what parts of it we want to give back," Rosing said.'"

Comments

and Sun is "opening" Java...

[garcia]

We have heard so much talk recently about the "open sourcing" of this piece of software and that piece of software that I just don't know if we should believe any of it anymore.

While Google is probably telling more of a truth than Sun is how do we really know until we see the code?

If a team is now just determining which code should be released we may not see anything useful come of this for months or even more. How about we hold off on these stories until we see something more than just a press release.

Re:and Sun is "opening" Java...

[Psiren]

While Google is probably telling more of a truth than Sun is how do we really know until we see the code?

You don't. But rather than just bitching about it, how about you just exercise some patience and wait a little while? Counting them out before they even get started is a little unfair, don't you think?

Re:and Sun is "opening" Java...

[koekepeer]

"Counting them out before they even get started is a little unfair, don't you think?"

yeah we only do that when we talk about MS or SCO or something *grin*

Re:and Sun is "opening" Java...

[koekepeer]

but google *is* good!

Re:and Sun is "opening" Java...

[Nikker]

Through an impossible mission I have **FOUND** the source code (hope google doesn't mind)
#!/usr/bin/perl .....
Oh wait!! bzzzzt

Re:and Sun is "opening" Java...

[Anonymous Coward]

"While Google is probably telling more of a truth than Sun..." I have to disagree. Sun already has things open source. Google is as locked up as Microsoft. Google is in violation of the GPL; they distribute their Google serach servers that use Linux without releasing the code, the vary thing we bitched to Linksys about.

Re:and Sun is "opening" Java...

[Kick the Donkey]

Actually, knowing how much google uses python, it would be more like:

#!/usr/bin/env python

:D

Re:and Sun is "opening" Java...

[lousyd]

Counting them out before they even get started is a little unfair, don't you think?

I think what the poster you're replying to was saying is: Counting them in before they even get started is a little unfair. "Going to release source code" doesn't necessarily mean they're going to flake out in some way, as you pointed out, but it also doesn't mean they're going to deliver.

Re:and Sun is "opening" Java...

[PigeonGB]

How about we hold off on these stories until we see something more than just a press release

No.

Re:and Sun is "opening" Java...

[standsolid]

those who can, do.
...those who can't, bitch about it on slashdot.

Re:and Sun is "opening" Java...

[Baricom]

Not if they're using a stock kernel. According to Linus, user-space programs aren't ever derivative works of the kernel, so its license "just doesn't matter."

IANAL, but it seems that the question is whether or not Google has made any changes to the kernel - to speed up searches, improve reliability, etc. I could be convinced either way.

Re:and Sun is "opening" Java...

[Red Alastor]

Even then, it would not matter. It is clearly written in the GPL FAQ that that if you don't distribute your software, you don't have to share the code. It can even be a trade secret.

Yahoo!

[xenostar]

In other news, Yahoo! announced it will be completely overhauling its search engine.

Re:Yahoo!

[gnu-generation-one]

"In other news, Yahoo! announced it will be completely overhauling its search engine."

Don't Yahoo use Google for their search results anyway?

Re:Yahoo!

[MikeCapone]

Don't Yahoo use Google for their search results anyway?

Not anymore, they dumped Google a few months back.

Will they release all or part of PageRank?

[Valejo]

Microsoft will smile, but pigeons will frown.

Re:Will they release all or part of PageRank?

[olethrosdc]

Pagerank has been described before. It's not a secret. Possibly they are using a fine-tuned version of it, but still...

Re:Will they release all or part of PageRank?

[arkanes]

It's described in the patent they have on it, of course, but what they actually use is pretty clearly not exactly whats in the patent.

Re:Will they release all or part of PageRank?

[j1m 5n0w]

It's also described in one of their research papers.

Link to patent.

-jim

Re:Will they release all or part of PageRank?

[mark-t]

How it works has been described, but to employ that method oneself requires crawling the web the same way that Google does.

There is no way to just _get_ a page's pagerank, as ranked by Google, without crawling the entire web yourself, or resorting to methods that are quite closely guarded secrets.

Re:Will they release all or part of PageRank?

[nacturation]

There is no way to just _get_ a page's pagerank, as ranked by Google, without crawling the entire web yourself, or resorting to methods that are quite closely guarded secrets.

You could always just install the Google toolbar into IE and it will tell you the PageRank number for the current page you're at. For example, the main Slashdot page is ranked 9/10. If you're feeling really adventurous, you could disassemble the toolbar code and find the API call to Google which returns this info.

Re:Will they release all or part of PageRank?

[Barryke]

I always thought pagerank was opensource already...??

Re:Will they release all or part of PageRank?

[olethrosdc]

How to crawl the web is hardly a secret, so I don't understand your point.

Google does use some interesting methods to crawl the web, but it is just a matter of managing resources, so that only sites that are updated frequently are crawled frequently.

In any case, if you want to have an algorithm that works, you can check out the Pagerank algorithm (without added safeguards for people that try to cheat, but those are just heuristics that are easy to add. You just have to fine tune them). There are other algorithms that you could use, which might be better than pagerank [depending on your purpose]. For example you could have an algorithm that weighs up incoming links, outgoing links and keywords in different ways, depending on what you think is most important.

However, you will have to fine tune the parameters of the algorithm for your particular application. If your application is searching the entire web and avoiding 'cheaters', then your parameters will be quite different from searching your database of documents.

This is the case for many algorithms that are well known. It is hardly a secret how neural networks work. However, even with source that implements one, you usually have to know a couple of facts related to their convergence, either from theory or from experimentation, in order to use them properly.

Which goes back to this: If google were to give you the source code for everything, how could you possibly expect to run it? If you would like to have something similar, you are encouraged to look at the Clever (IBM's similar algo) and Pagerank papers, implement the algorithms, play around with them in some small database (like your own website) and then try to apply them to the problem at hand until you get a satisfactory performance.

Re:Will they release all or part of PageRank?

[mark-t]

Did you actually _read_ what I said before posting?

I said that how pagerank works is widely known. The algorithm is actually very straightforward and can be written by anyone with a modest amount of programming skill in short order. Unfortunately, to actually do it oneself requires crawling the entire web the same way that Google does. This isn't unfortunate because nobody else knows _how_ to do this (it's trivial), it's unfortunate because of sheer volume of time and other resources it would take to actually do it. Most people do not have access to that level of resources, so using the pagerank algorithm oneself is simply infeasable.

Re:Will they release all or part of PageRank?

[olethrosdc]

I did read and I was not sure I understood your point, so I replied and you have now clarified. You had said:

> How it works has been described, but to employ that method oneself requires crawling the web the same way that Google does.

Valid, point. However, if (as the OP) you are talking about an entity that *could* crawl the entire web, then it is not applicable.

> There is no way to just _get_ a page's pagerank, as ranked by Google, without crawling the entire web yourself, or resorting to methods that are quite closely guarded secrets.

I was a bit intrigued by this sentence's meaning, especially the phrase 'closely guarded secrets'. That's why I said 'hardly a secret'.

Re:Will they release all or part of PageRank?

[mark-t]

The closely guarded secrets to which I was referring is how to extract a page's pagerank directly from Google itself. After all, their pagerank plugin for IE does it, so it's possible... but how their tool does this is not published. Some organizations have claimed to have figured this out and are charging a pretty penny for their results. The pagerank toolbar can be reverse engineered, but the efforts aren't really justified since Google upgrades it every few months anyways.

They were here ...?

[The Sith Lord]

A recruiting drive in Melbourne?
Why wasn't I informed? (nice one RMIT, keeping us students in the know)

Re:They were here ...?

[Anonymous Coward]

I heard about it through the grapevine only... And I am a sysadmin at the uni it was held. I think they were trying to keep it hush hush in order to sort the snotty undergrads from the people they were potentially interested in employing - researchers. Come back when you get your Ph.D kid...

BTW, a friend who went to the career night said that they were looking for these things in order; extremely high intellegence, fit with their culture, programming skills desirable but optional..

Re:They were here ...?

[SEWilco]

Why wasn't I informed?

What, you didn't Google for "Melbourne recruiting drive" like everyone else did?

Give something back?

[slavefishy]

I whole-heartedly support open source and I think it's great they're going to reveal some of their code, but I don't feel that Google owes the community anything; they already provide top quality services.

Re:Give something back?

[WormholeFiend]

but I don't feel that Google owes the community anything; they already provide top quality services

Au contraire, mon ami poisson esclave.

Google wouldn't exist if it weren't for people creating/putting content on the Internet... (or, if you belong to the tinfoil hat crowd, putting your personal data online).

But it may not be so much a "debt" relationship as it is a symbiotic one between Google (and other search engines) and Internet users... without content, no search engine... without search engines, hard-to-find content.

Re:Give something back?

[Anonymous Coward]

I think they do owe the community a lot. They are breaking the spirit of GNU. The idea behind open source is that if I choose to publish my stuff under GNU, you can take my code, work with it, sell it, but I get your improvements and use that myself. This works great if software is something that you buy or download, but it doesn't work if it is a service like google. If I was say a coder on an open source IMAP client and google used my code as a basis for gmail, they could keep all the code for themselves. Google is using a lot of open source software but they are not giving any of there improvements back to the community. I know Google is nice and all, but as software gets more service oriented this could be a problem in the future.

Re:Give something back?

[geminidomino]

I think they do owe the community a lot. They are breaking the spirit of GNU. The idea behind open source is that if I choose to publish my stuff under GNU, you can take my code, work with it, sell it, but I get your improvements and use that myself

Who gives a rats ass? Now they can't release ANY source without inflicting it with that hideous excuse for a license, for fear of "breaking the spirit of GNU?" Or are you accusing them of breaking copyright law by using GPLed code in closed source (as opposed to internal-use only) projects?

For all the whinging the GPL-clerics like to do about things being "owed," you sure seem to be a bunch of ungrateful fucks.

Re:Give something back?

[cabjoe]

They've already given back more loads than they've taken. I remember reading the article the authors wrote in Scientific American (either June or July 1999), trying it out and immediately finding that they were giving me much better results than the best of the best of the time (Metacrawler, altavista etc., etc.,) were. For a few months I felt that I was one of very few in on an amazing secret.

So what if they used GNU software to power their engine. Exactly what part of their search algorithm depends on GNU software?

The fact is that for the last five years I have used Google exclusively for web searches and I can count on the fingers of one hand the number of times I clicked though one of their ads. I owe them loads, they owe me nothing. Good luck to them and thanks again for all the results (whoops nearly said fish there).

The Code - REVEALED!

[strictnein]

Preview of source code to be released:

int main()
{
// magic google code removed

// Rest of code

// How many records did we "find"?
int random1 = rand();
int random2 = rand();
int random3 = rand();
int random4 = rand();

cout << "Results 1 - 10 of about " << random1 * random2 ^ random3 << "for " << SEARCH_ENTRY << ". ( " << random4 << "seconds)";

}

Re:The Code - REVEALED!

[Anonymous Coward]

Yahoo has decided to release some of theirs:

std::vector mainsearch( const std::string& pQuery )
{
if (pQuery == "George Bush")
return DoBeltway();
else if (pQuery == "Osama Bin Laden")
return DoTerror();
else if (pQuery == "JLo")
return DoCelebs();
else if (pQuery == "Microsoft")
return DoTech();
else if (pQuery == "Lord of the Rings")
return DoGeekFlicks(); // else
return DoAds();
}

Re:The Code - REVEALED!

[Psiren]

You haven't flushed your output ;-)

Re:The Code - REVEALED!

[Issue9mm]

That's how the results keep growing!!!

-9mm-

Re:The Code - REVEALED!

[beattie]

WOW! An int() function doesnt even have to return anything. That IS magic!

Re:The Code - REVEALED!

[OverflowingBitBucket]

It's not enough that Google are going to reveal some of their code, but people expect them to give away their special return value to main() too? *shakes head sadly*

Re:The Code - REVEALED!

[NeoThermic]

I've also managed to get the pre-search code:

<html><head><meta http-equiv="content-type" content="text/html; charset=UTF-8"><title>Google</title><style><!--
b ody,td,a,p,.h{font-family:arial,sans-serif;}
.q{c olor:#0000cc;}
//-->
</style>
<script>
<!--
f unction sf(){document.f.q.focus();}
// -->
</script>
</head><body bgcolor=#ffffff text=#000000 link=#0000cc vlink=#551a8b alink=#ff0000 onLoad=sf()><center><table border=0 cellspacing=0 cellpadding=0><tr><td><img src="/images/logo.gif" width=276 height=110 alt="Google"></td></tr></table><br>
<for m action="/search" name=f><script><!--
function qs(el) {if (window.RegExp && window.encodeURIComponent) {var qe=encodeURIComponent(document.f.q.value);if (el.href.indexOf("q=")!=-1) {el.href=el.href.replace(new RegExp("q=[^&$]*"),"q="+qe);} else {el.href+="&q="+qe;}}return 1;}
// -->
</script><table border=0 cellspacing=0 cellpadding=4><tr><td nowrap class=q><font size=-1><b><font color=#000000>Web</font></b>&nbsp;&nbsp;&nbsp;&nbs p;<a id=1a class=q href="/imghp?hl=en&tab=wi" onClick="return qs(this);">Images</a>&nbsp;&nbsp;&nbsp;&nbsp;<a id=2a class=q href="/grphp?hl=en&tab=wg" onClick="return qs(this);">Groups</a>&nbsp;&nbsp;&nbsp;&nbsp;<a id=4a class=q href="/nwshp?hl=en&tab=wn" onClick="return qs(this);">News</a>&nbsp;&nbsp;&nbsp;&nbsp;&lt ;a id=5a class=q href="/froogle?hl=en&tab=wf" onClick="return qs(this);">Froogle</a>&nbsp;&nbsp;&nbsp;&nbsp;<b>< a href="/options/index.html" class=q>more&nbsp;&raquo;</a></b></font></td></tr> </table> <table cellspacing=0 cellpadding=0><tr><td width=25%>&nbsp;</td><td align=center><input type=hidden name=hl value=en><span id=hf></span><input type=hidden name=ie value="UTF-8"><input maxLength=256 size=55 name=q value=""><br><input type=submit value="Google Search" name=btnG><input type=submit value="I'm Feeling Lucky" name=btnI></td><td valign=top nowrap width=25%><font size=-2>&nbsp;&nbsp;<a href=/advanced_search?hl=en>Advanced&nbsp;Search</ a><br>&nbsp;&nbsp;<a href=/preferences?hl=en>Preferences</a><br>&nbsp;& nbsp;<a href=/language_tools?hl=en>Language Tools</a></font></td></tr></table></form><br><br>< font size=-1><a href="/ads/">Advertising&nbsp;Programs</a&g t; - <a href="/services/">Business&nbsp;Solutions</a&g t ; - <a href=/about.html>About Google</a></font><p><font size=-2>&copy;2004 Google - Searching 4,285,199,774 web pages</font></p></center></body></html&g t ;

(You'll have to forgive /. for playing about with some of it)...

NeoThermic

Re:The Code - REVEALED!

[toddestan]

That's the secret!?! And all those hours I spent googlewhacking...

Re:The Code - REVEALED!

[Electrum]

const std::string& pQuery

Why prefix a reference with a p (pointer)?

Re:The Code - REVEALED!

[alerante]

Wait ... if that was a standard rand() function, it either returns a minuscule amount of search results (0.00347) or a l-o-o-o-n-g time ("I waited 15 #$&%ing DAYS for this?!").

So it surely can't be that.

Re:The Code - REVEALED!

[Ninwa]

According to standards main() isn't actually a function, and you don't HAVE to return a value. If you don't return a value EXIT_SUCCEESS is assumed. And you call yourself a /.'er ... I cry for you.

give back?

[OmniVector]

Google's done so much for the quality of web searching that I think they've already given us far more back than I could ever ask for. The only thing I could ask of them is to keep up the good work, and try and keep pagerank useful by stopping exploiters who falsely boost ranks. Other than that, some good old free thinking and continual innovation for the web, which is what they do all the time, is little to ask for. I just hope the momentum doesn't die.

Re:give back?

[Otter]

To put it a different way (and this isn't directed specifically at you, just a general observation consistent with what you said) -- the people who do most of the screaming about "Free! Free! You owe it to The Community after everything We've done for you!" have no interest in or use for source code themselves. They just want something for free. See any Apple article for examples.

Everything from Google already is free (beer). If they decide to open some code, I'm sure it will be fascinating reading for CS people interested in searching or scalability, but the noisy "community" couldn't care less.

Re:give back?

[AviLazar]

I don't necessarily see that as "giving" They are a for profit organization. While they provide a service that is 'free' to the end-user, it is still paid for by advertisers. The fact that they are the best at it does not elevate them to a status of saints (for lack of better words).
Depending on the code they will show (assuming this is not a lame stunt) they will actually be "giving back" to the community.
Otherwise, they can open their fat pocket books and make donations like everyone else :)

Re:give back?

[kabrakan]

This is very cool because Google uses extremely advanced AI, which is testament to how this field is very commercially viable. I don't know what exactly they will reveal, but it should teach a thing or two to the AI community.

Re:give back?

[Tony-A]

have no interest in or use for source code themselves. ... the noisy "community" couldn't care less. >/i>

Hmmm, I'd expect bits and pieces of the code, and the mind-set that accompanies them, to show up in lots of strange places. A good web search is fine but I suspect Google's real value will be in corporate infrastructure.

Re:give back?

[krgallagher]

"Google's done so much for the quality of web searching that I think they've already given us far more back than I could ever ask for."

While this is true and I whole heartedly agree, Google is not being completely philanthopic here. Here is a quote from the original article:
"He says it isn't fair for Google to draw smart people from all over the world and "just keep it all for ourselves. We need to have the tools out in the universities so the next generation can build on our work, too."

See, Google gets it. Since they are part of the community, doing good for the community equates to doing good for themselves.

Re:give back?

[Tony-A]

Bah, flumble flingers.

Re:give back?

[iabervon]

The community they're giving back to is not the people who search the web, but the people who develop software they use. Of course, practically anyone who develops software searches the web, but the service isn't really a repayment in kind.

In addition to their web searching engine, they have a whole lot of interesting software. Their PDF to HTML converter, for example, is nicer in some ways than any PDF viewer I've seen for Linux. They clearly have some useful tools for transforming HTML, as well (highlighting words, e.g.). That's the sort of thing that's nice to release to the community, both because people will like it, and because people will improve and maintain it.

Actually, the best benefit could be the simple fact of the code and therefore capabilities being the same: consider the effect it would have on web publishing if Google searches would only find a Word document if Open Office could read it, and FireFox would render DOC the way Google sees it. IE has a large market share, but Google's is possibly even more strategic. There are lots of good possibilities here: if FireFox can't skip your flash intro, Google won't either, and you won't get any hits. If Open Office mangles your document, Google will also mangle it while highlighting the search terms.

Re:give back?

[gnu-generation-one]

"They are a for profit organization."

According to the IPO papers, they're a "not for evil" organisation... It even mentioned that profit wouldn't be their core interest.

Re:give back?

[AviLazar]

Profit is not their core interest? But it is one of their interests? I worked for a bank, and their mission statement that their most important priority was customer service...I am pretty sure the COMPANY's most important priority was making a buck.
A company that wants to make a buck does not qualify as an "evil" organization. Google is a for-profit company - again let's not elevate them to saints because they happen to be popular and have a clean reputation (so far).
Don't get me wrong, I do like the Google search engine and I wish them the best. But I am realistic and do realize they are here to make money - otherwise they wouldn't charge more then the bare minimum needed to keep their servers running and their employees paid at an average level.

Re:give back?

[Stallmanite]

One reason the community doesn't care as much about google being free is because its a service, not code being executed locally; google has less potential for malice then an installed program.

IMO the people shouting "free" aren't asking for things for $0. If that is all they are after "piracy" works just as well. IMO they are asking for peer review, no vendor lock in and life for programs after the authors abandon them.

Re:give back?

[nacturation]

I worked for a bank, and their mission statement that their most important priority was customer service...I am pretty sure the COMPANY's most important priority was making a buck.

No, it really is customer service that's the most important. The reason why is because it's the happy customers who keep coming back and recommending the bank to their friends that results in the bank turning a buck. If they truly put profits first at the expense of customer service, their customer base would dwindle as would their profits.

Re:give back?

[AviLazar]

No, it really is customer service that's the most important. The reason why is because it's the happy customers who keep coming back and recommending the bank to their friends that results in the bank turning a buck. If they truly put profits first at the expense of customer service, their customer base would dwindle as would their profits.

The banks first priority is to make a buck - to make this buck they need to do things, one of which is make the customers happy. The people who work for the bank (from the CEO to the part-time teller) are all there to get paid. The company's shareholders have shares to get paid. The company's biggest desire is to make more money so it can survive. To sugar coat it by saying their biggest concern is customer happiness is either nieave(sp?) and/or corporate bs.

Ok let me put it in other words - if the banks biggest concern was customer happiness, then the bank would give each customer a 0% loan, no matter their credit rating. They would give each person a free checking account that gives them free checks, internet access, etc. DOesn't charge them any fees and pays all bounced checks. The bank doesn't do this, why? Because it is not profitable. That is all there is to it.

Re:give back?

[nacturation]

Ok let me put it in other words - if the banks biggest concern was customer happiness, then the bank would give each customer a 0% loan, no matter their credit rating. They would give each person a free checking account that gives them free checks, internet access, etc. DOesn't charge them any fees and pays all bounced checks. The bank doesn't do this, why? Because it is not profitable. That is all there is to it.

You're right, in a way. But a customer who expects free service from a company is mistaken. Some restaurants offer a satisfaction guarantee, saying that they promise you'll be satisfied with your food. Does this mean that customers are only satisfied if the meal is free? Of course not. People expect to pay for a service... whether it's banking or getting a meal served to them. Within that expectation of payment for services, the customer can still be very happy. But once a bank crosses the line of charging reasonable money for services and they start screwing over the customer, then a happy customer becomes bitter and will take their business elsewhere.

I know what you mean though -- it's somewhat of a sham to say that any business wants a happy customer. What they really want is money in their pockets. But in order to get more money, it requires more customers doing more business. So businesses are really forced to make customers happy. If they could charge you money and you're required to pay no matter what, then they'll gladly screw you over. An example of this is when you pay income tax. The government couldn't care less if you're happy about it. You're required to do it and that's that, so the government happily screws you over.

Re:give back?

[AviLazar]

Agreed. Unfortunately our gov't is a monopoly ;) Hmm anti-trust suit anyone?
In all seriousness, i am sure businesses would rather have happy customers (not even considering keeping them as customers). A happy customer doesn't complain, complaining involves listening, listening requires time, time costs money...oh ;)
I firmly believe the world is more positive then negative, so I believe that companies do want their clients happy, but since I am not oblivious to things I also realize people work to make money, and anyone who works because it is fun apparantly already has money (and probably too much of it, so fork it over) ;)

Re:give back?

[superyooser]

Depending on the code they will show (...) they will actually be "giving back" to the community.

Google can't be giving code "back" to the community since the community never owned or helped create Google's code. Google can give us their code, but it cannot be said that they are giving it back. It was never ours. We are not entitled to it.

Re:give back?

[AviLazar]

Google can't be giving code "back" to the community since the community never owned or helped create Google's code. Google can give us their code, but it cannot be said that they are giving it back. It was never ours. We are not entitled to it.

I never said they were giving back code; I said they will be "'giving back' to the community." The statement "Depending on the code they will show ... they will actually be 'giving back' to the community" does not say they are giving code back. I know, as well as most of us, that the code was not created by the open source community.
Stupid nitpicking (dissecting) like this serves ZERO purpose especially when you did it wrong.

Excellent!

[r1ch]

Although Google obviously won't be releasing it's search algorithms it might well release the code for things like the Google FileSystem (PDF) which may benefit a lot of people.

Re:Excellent!

[SpiritOfGrandeur]

Although Google obviously won't be releasing it's search algorithms it might well release the code for things like the Google FileSystem (PDF) which may benefit a lot of people.

If this was to happen just imagine how much longer we would have to wait for Microsoft to release Longhorn.

Re:Excellent!

[LiquidCoooled]

The actual source surely isnt actually *that* important. It might have been way back in the beginning, but not so much anymore.

I'm quite convinced that the code itself is relatively simple, each node handling its own small piece of the puzzle.

It would take years before anyone actually making use of the code could build up the infrastructure and reputation that google has got, in the meantim,e we could make some seriously funky projects out of it.

I would love to be able to incroporate google search algorythms and procedures into (for instance) an SQL query, and allow searching of the myriad of OFFLINE data we have here.

"select (feeling_lucky) from customerrecords ..."

At the very worst, the code becomes an academic curiosity, at the best, googles algorythm becomes as well adapted as Huffman coding or the bubblesort.

Re:Excellent!

[ses4j]

Although Google obviously won't be releasing it's search algorithms it might well release the code for things like the Google FileSystem (PDF) which may benefit a lot of people.

I totally disagree. I think the GoogleFS is a much more valuable commodity than the search algos. I mean, frankly, I doubt think the search algos are that brilliant, past the initial lightbulb of PageRank... just refinements and optimizations. The tough part is harnessing the -insane- computing power necessary to serve the world's searching needs, and doing it cheaply.

Despite that, I do hope you're right, and maybe you are... since the distributed FS/OS they've developed is, like I said, so much more valuable. What good would search algorithm descriptions do anyone except aid their competition? I can't stick Google's algorithms into anything I have... but a nifto OS that can combine a few computers and let me run stuff across them trivially? -THAT'S PRETTY COOL-

Re:Excellent!

[ecklesweb]

The actual source surely isnt actually *that* important.

I agree. What would be cool is if they released the design and some details around how and why the algorithms work. With that, you could implement in any language. Google releasing just the source means that someone (manyones actually) are going to have to spend a lot of time reverse engineering the code to come up with why the thing works.

It would take years before anyone actually making use of the code could build up the infrastructure and reputation that google has got

That's a good point, and is probably the best business-based insight: Google's competitive advantage has nothing to do with the code they are planning on releasing. If it was, they wouldn't release it. If it was and they did, that IPO isn't going to be worth too much. I think you're correct that infrastructure, integration, market share, etc. are items on the list that comprise Google's competitive advantage. Code isn't going to be (high) on the list.

Too bad I can't reply to a post and mod it up.

Another funny thought just came to mind: how well do you think Google developers comment their code?

Re:Excellent!

[GeorgeH]

Didn't they publish their search algorithms in Patent 6,285,999 "Method for node ranking in a linked database"? That's the PageRank algorithm; since it's patented it's publicly documented and available for public use 21 years later.

Re:Excellent!

[Anonymous Coward]

Python? Google has PHDs working for them. Obviously the engine is written in Perl you fool.

Re:Excellent!

[travail_jgd]

"It would take years before anyone actually making use of the code could build up the infrastructure and reputation that google has got"

Let's say a company wants to buy their way into the search engine business...

If each PC cost an average of $3000US (to include large amounts of RAM, networking infrastructure, etc) 50,000 of them would cost a "whopping" $150 million. That's a lot of cash for a startup, but pocket change for Apple, IBM, or Microsoft.

Both IBM and Microsoft have the programming resources to take advantage of the hardware, although IBM would have the upper hand (IMHO) having an army of consultants that they can draw upon for technical help.

Re:Excellent!

[drinkypoo]

but a nifto OS that can combine a few computers and let me run stuff across them trivially? -THAT'S PRETTY COOL-

Yes, it is, and it's called OpenMOSIX.

Re:Excellent!

[KevinKnSC]

Yeah, especially if by "cheap PC" you mean "some PC components on a shelf".

Re:Excellent!

[PhrostyMcByte]

Agreed. It takes a true Pigeon-harvesting dog to understand Perl.

Re:Excellent!

[JustinXB]

"...but a nifto OS that can combine a few computers and let me run stuff across them trivially?" They have that: Plan 9, Inferno, and Amoeba.

what next?

[KrisCowboy]

The best search engine...one of the best translators, damn good webmail...what next? Guess it's time for gmessenger!!!

Re:what next?

[hey]

It would be great if Google would adopt and popularise Jabber.

Re:what next?

[cipher uk]

yeah i was thinking about a gmessenger. i mean they already do everything else that msn does .. just better.
well except for msn explorer .. google are intelligent to know what a product firefox is :). oh and msn as an isp.
they could really stamp on msn's feet with a gmessenger.

Re:what next?

[beattie]

At what point does the Google g prefix start to encroach on the GNU g prefix? I could imagine someone thinking gmail was the gnome/gnu equivalent of kmail.

Re:what next?

[Dogers]

someones already apparently started this
http://cgi.ebay.com/ws/eBayISAPI.dll?ViewIte m&cate gory=4291&item=3683465610&rd=1&ssPageName=WDVW

whether or not thats rubbish i dont know, but hey :)

Re:what next?

[PsychoKiller]

Why not? It could scan what you are typing about to a friend and display relevant text ads in a sidebar.

Ie:

1) What are you doing tonight?
2) I dunno, I was thinking about seeing a movie. Want to come with me?
1) Sure, want to go see Shrek 2? - At this point a text ad comes up in the sidebar with local movie listings.

I've been using gmail for about a week now and I've found that their ads are surprisingly relevant to my conversations. One time, someone mentioned 'Miss Manners' in an email and an ad came up to buy a copy. I think they'll get a lot of click-throughs on their email this way, and it could be extended to an IM concept.

Re:what next?

[RPoet]

Isn't just about all of Europe around 80% MSN, and probably rising? MSN seriously needs a serious contender, and Jabber + Big Corp is the only one that can grant it.

Google knows how to build communities, just look at Orkut which just passed 500 000 members the other day, and gains 10 000 new members every day. Popularize Jabber for us, Google!

Re:what next?

[sik0fewl]

Damn, beat me to it.

Anyway, it would be great if Google came out with it's own instant message program using Jabber. Google is the "cool thing" right now and it would be a lot easier to convince my friends to use "google messenger" than some thing called "Jabber". I wouldn't have to tell them it's open and free and therefore better. I can just tell them "google is cool, use it".

Re:what next?

[metalogic]

My bet is on Gauction, leveraging their expertise in data-mining for fraud detection.

Re:what next?

[lewp]

Since the number of people who know about GNU/GNOME/Kmail is already several orders of magnitude smaller than the number of people who know about gmail (something they can't even sign up for yet), does it really matter?

Anything associated with Google already has far more name recognition with the general public than any OSS project with the possible exception of Linux itself.

My mom, one of the least technical people I know, called me the other day to tell me how great she thought Froogle was. She also knows about (and is looking forward to joining) Gmail. She's been a Google devotee on the search engine front for the last year or so.

If Google ever (accidentally, I'm sure) stomped on the name of an OSS project I imagine it would be the OSS project that would have to get out of the way.

Re:what next?

[sewagemaster]

MSN seriously needs a serious contender, and Jabber + Big Corp is the only one that can grant it.

actually i personally think good old ICQ still beats MSN messenger hands down. it was still rocking until ICQ+BigCorp (AOL) took over (, which happens unfortunately to all the apps/companies that AOL takes over (icq, nullsoft/winamp, netscape).) ICQ is turned bad because the recent clients are ultra-bloated and filled with ads. even trillian is slow. The best app i still believe for the protocol is LICQ under *nix.

There's also no offline messessging in MSN.

Only thing I dislike about jabber is that the servers are always down.

Re:what next?

[Psychic Burrito]

I'm hoping for gPIM, with email, address book, calendar, notepad, a contacts-db and a small word proc, all integrated and ready to use: Click a person and see all the emails to sent and received from him, your phone calls, meeting notes and much more. This would rock!

And I'm also hoping for gWiki, where small organisations can just click twice and get their own wikimedia to play with.

Anybody agrees? Say "yeah"! :-)

Forget about search engine code

[Karl Cocknozzle]

What I want to see is whatever mods they've created to make managing the enormous uber-cluster(s) that make the place tick. Plus, more than likely, they won't reveal the search code anyway...

But I salivate to review the code to their management tools.

Re:Forget about search engine code

[swb]

I have a friend who works there who I just saw for the first time since college. I asked him what he's worked on since he's been there (you can count to his employee number on your fingers and toes), and he said one of the things was internal code to optimize the bandwidth of links.

From our conversation, I suspect that a lot of the really hard work is optimizing their use of networks and clustering. PageRank is a neat idea, but without an efficient distributed clustering system to make it work you run into problems.

Re:Forget about search engine code

[thogard]

Their cluster managment is based on the fact that any node can fail at any time and a few thousand users will click reload. Thats not too useful for most other real world clusters.

Re:Forget about search engine code

[jovetoo]

I agree. It would, for example, be interesting to see how they can keep all their databases synced all over the globe or if they pass on request between sites.

The search code they might publish, it is detail on page-ranking (and related techniques) that are not going to be distributed. (It would allow too many people to manipulate these ratings and it is their competitive edge...)

Personally, I think this is a very good move of Google. In the long run, it is more likely that Google will "be getting something back" however. That is, if they have the guts to publish enough of their code for people to be able to participate.

Search technology is still in its infancy. Afterall, it is little more than a glorified grep on a huge database. It does not understand about context, topic, concepts or even text purpose. (For example, a combination of a moderation system with bayesian filters could most likely be used to differentiate between technical documentation and marketing fudd.) There is *lots* people can contribute.

Just don't help the optomizers

[weave]

I'm sure google isn't stupid, but I'd hate for them to reveal anything that the search engine "optomizers" will leverage to further spam the search results.

What code ?

[Anonymous Coward]

A Google Toolbar running on Linux would be a good start...

Re:What code ?

[farzadb82]

Why not use the google bar that comes with Mozilla/Firefox ?

Re:What code ?

[tukkayoot]

Is it really necessary? I wouldn't complain, of course, but most of what I used the Google toolbar for with Internet Explorer is done by Mozilla and Firefox already.

Re:What code ?

[guybrush876]

Don't need the code for that
http://googlebar.mozdev.org/

Re:What code ?

[consolidatedbord]

Q: Why not use the google bar that comes with Mozilla/Firefox ? A: Because it doesn't support the page ranking feature.

Re:What code ?

[blackmonday]

Download Firefox and enjoy.

Business Strategy

[DarkClown]

They're business strategy flips me out the more I learn about it.
Wish I had a crystal ball, or a crystal search engine to see where they're going to go with their 'relevant content' model next. Is there a hole in their approach? Seems pretty strong! Will be intersting to get a look at their financials after public offering.

Good ol' google

[LaserLyte]

I think the average web user would be pretty surprised to see what actually goes on behind the scenes at Google. That simple search page which holds very little content -- the search controls, a logo, and a few links -- is actually a front to a mighty army of servers, and some fairly advanced search technology.

I for one know I often take Google for granted. Yet I use it many times, virtually every day. In return for a free service, I get to see a few tiny, highly-revelant unintrusive ads placed alongside my search.

The only improvement for Google I can think of is for them to add regex searches :D. A lot of pitfalls and obstacles come to mind for that though.

Anyway, I can't wait to see what Google actually "gives back to us" in terms of source code. Can they give enough source to be interesting/useful without helping out their competitors too much?

Re:Good ol' google

[craigmarshall]

$10 dollars for 1000 (or 10,000 or whatever) searches? I think that could work.

Craig

Re:Good ol' google

[ari_j]

The only improvement for Google I can think of is for them to add regex searches :D. A lot of pitfalls and obstacles come to mind for that though.

I'd like to see proximity searches, like "Jane within 5 words of Doe" or "atomic within the same paragraph as mutants". Too often, I am left wanting a compromise between searching for an exact phrase and searching for the words in that phrase, and not getting the correct results.

Re:Good ol' google

[johndeeregator]

Reno 911! Dangle: Jones, run his name through the computer. Jones: Should I Google him? Dangle: Google? You have a f--king police computer. You don't need to use f--king Google. You're spending too much time looking for Internet porn.

Re:Good ol' google

[AviLazar]

I think the average web user would be pretty surprised to see what actually goes on behind the scenes at Google. That simple search page which holds very little content -- the search controls, a logo, and a few links -- is actually a front to a mighty army of servers, and some fairly advanced search technology.
The "average" web user couldn't tell the difference between a line of code and a two year old's scribbling
"Cout? What's a cout? Boy, you've made a typo - you meant to say clout. You know, what politicians use to get what they want." :)

Re:Good ol' google

[thogard]

Altivista has a "near" keyword that does what you ask. Its a great way to fix the serches of people who put too much unrelated crud in their menus that google finds.

Re:Good ol' google

[darien]

I'd rather pay $10 a year for unlimited searches. Often my first few searches just turn up Kelkoo-type crap, and it takes a few goes to come up with a working query. That's annoying enough as it is, without my having to pay for it!

Re:Good ol' google

[Khalid]

Google already take care of the "near" but in a celver way that Altavista does, for instance if you search for : foo bar

"foo bla bar" will rank better than "foo blah blah bar" as in the first sentence foo is more "near" bar than in the second !

Re:Good ol' google

[LS]

"In return for a free service, I get to see a few tiny, highly-revelant unintrusive ads placed alongside my search."

Umm, how much did google pay you? [**ducks**] as google worshippers throw their shoes at me :)

"Back"?

[InternationalCow]

Giving something back implies that they took something from the users (?) and are now paying back. So, what did they take? Is there F/OSS code in their code? Or do they just want to thank the users for using their services? In that case, I'd take a Gmail account, please :)

Re:"Back"?

[phurley]

Well it is common knowledge that they run their system off of (lots and lots of) Linux servers. So maybe that is what they mean by "give back"

Re:"Back"?

[BabylonMink]

The Google cluster runs Linux. An army of people created Linux and Google get to download and use it for free. I am positive that there are other OpenSource tools in use at Google in other areas as well. These are the things Google 'took'. I prefer the term 'made use of'

Re:"Back"?

[FauxPasIII]

Also, from python.org:

"Python has been an important part of Google since the beginning, and remains so as the system grows and evolves. Today dozens of Google engineers use Python, and we're looking for more people with skills in this language." said Peter Norvig, director of search quality at Google, Inc.

Re:"Back"?

[Horia]

They take the content from my website, your website and everyone else's, they request sometimes thousands of pages per day, consume everyone's CPU cycles, bandwidth, etc to build their business. So yes, they do take, they take a lot! And they don't pay us for the bandwidth used by their bots.

In fact except for the keyword crunching system, everything they use is taken from the rest of the world.

Put Google on a internet with no websites and it has no value of itself.

Just my .02

Re:"Back"?

[cheese_wallet]

"Giving something back implies that they took something from the users (?) and are now paying back. So, what did they take?"

If I give you my good will and do a good deed for you, have I lost anything? Not always, and in some cases I might even gain something from it (peace of mind, contentment, etc...). Have you gained something? Yes. And perhaps you would like to return the favor.

Google has gained the good will of the public. We like them, and encourage them, and use their service. They have gainful employment, a mental playground for their employees, and all manner of blessings. I think it shows something of the character of google management that they would want to give something back. Especially since they are at the pinnacle of the search engine market, and could comfortably sit on their laurals if they wanted to and still be at the top.

btw, I sent you an invite to gmail.

Re:"Back"?

[InternationalCow]

Just wanted to say THANK YOU to all the friendly /.'ers who mailed in offering a gmail invite! I of course accepted one but cannot accept all of them (although I was tempted :) ). So, thanks again for the friendliness and generosity, it is greatly appreciated. Cow

Re:"Back"?

[iMaple]

All of us working at google totally agree with you. As a token of our agreemennt and thnaks we have decided to extend you an invitation for a new type of an email account. You get a useful 2Mb instead of a measly 1Gb account space. And unlike the self proclaimed geeks who claim they beta test the Gmail software and dont even get a single bug after 4 months of use, we promise that you will be using a webmail program which will be eternally buggy. Also if you sign up really soon(within the next 300 years) you will get a daily advertisement for penis enlargement totally free !!!!!

So click below to get your complimenatry invite. Hotmail

Re:"Back"?

[plaa]

Giving something back implies that they took something from the users (?) and are now paying back. So, what did they take? Is there F/OSS code in their code?

Just before anybody starts screaming code theft (and I'm not saying you did - it's just easy to get that picture from the above comment), it would be totally legal for Google to use any GPL code in their search system without publishing it. The GPL requires you to make the source available to anybody they distribute the program to. As they don't distribute their search engine code (I'm not counting things like the Search Appliance), they have no obligation to publish the modifications, either.

This is the whole point of the GPL - you're allowed to do anything you wish with the program, but if you distribute a modified program, you must give the receiver the same rights.

Re:"Back"?

[burns210]

"Giving something back implies that they took something from the users (?) and are now paying back."

Well, maybe he meant 'give back' in the sense of repaying Google for providing a service that has benefited internet use globally, while offering it for free and being unbiased in their results... Maybe he thought that that service was such a kind act by a private company, that Google deserves to be 'thanked' in a significant way.

why do they have to give something back?

[chrisopherpace]

I love Google, and think this is an excellent idea, but geeks owe Google, not vice-versa. Geeks (and normal users as well) depend on Google. While it could be said that Google does the same, and makes its money from geeks and users, they do not, IMHO owe us anything.

Re:why do they have to give something back?

[chrisopherpace]

Google already has given Linux users plenty. Think of all the newbies that had a problem, and Google helped. How many of those newbies would still be using Linux today? Google has saved my arse I don't know how many times, where other search engines came up with nothing. That all being said, I think it is one hell of a good show on their part.

Re:why do they have to give something back?

[jokerghost]

They don't "have to" give anything back, but the idea is that they are acting altruistically. They feel that the public has supported their company (via usage, interest in the IPO, jobs, income, etc), that they recipricolly owe something substantial to the consumer.

Read a little Ayn Rand (like Atlas Shrugged) for the concept... Essentially, corporations (whether private or not) derive their status as a legal "person" from implicit trust the public has placed in them. As such, corporations have an obligation to provide service back to the community and to act in a moral fashon... I for one am glad to see that this company is acting in this accord, instead of all of the depravity we have seen as of late.

-jokerghost

Re:why do they have to give something back?

[burns210]

"but geeks owe Google, not vice-versa. Geeks (and normal users as well) depend on Google."

so buy some stock (when they go public) and move on, not that hard...

how?

[Janek Kozicki]

how they will manage to do that without shooting themselves in the foot?

Re:how?

[tukkayoot]

By aiming very carefully.

Re:how?

[fanfriggintastic]

Having a friend inside Google, I'm privy to certain information. The first few lines of the google code, for example:

<HTML>
<HEAD>
<TITLE>Google </TITLE>

...

Re:how?

[mgcarley]

with all those CAPITALIZED TAGS, it wouldn't be XHTML compliant!

Odd

[sockman]

I don't think Google needs to 'give' back since no one "pays" for their service (I don't pay for news or search or gmail, but the ads I don't follow do).

Perhaps this will boost OSS a notch, if a company like Google can contribute, why can't other companies?

Nothing "magic" about it...

[192939495969798999]

They already showed the algorithm they use for Pagerank, and let's see... they have thousands of systems all hooked together, that process terabytes of data every month. That sounds more like hard work and money than magic. The magic will be how they are keeping Microsoft at bay.

Re:Nothing "magic" about it...

[Ford Prefect]

The magic will be how they are keeping Microsoft at bay.

I'd say they've laid down the gauntlet, and are seeing what Microsoft does next.

Imagine what Microsoft would feel like, in possession of Google's crown jewels - countless millions of lines of Linux-dependent source code. It would be bloody expensive to port, and if the only way of running it is by installing Linux on zillions of servers...

Go on, Microsoft, give in to the dark side! ;-)

Re:Nothing "magic" about it...

[Iaughter]

If you read the article, you'd see that they're talking more about encouraging their army of PhD's to publish their research than open-sourcing whatever proprietary code they have.

Re:Nothing "magic" about it...

[interiot]

Eh? Where's the source code to PageRank? Was source code released to reflect the multiple recent changes to it?

I'm guessing the answers are "no" and "no". SEOs (search engine exploiters) would only damage google if google gave out their source code for pagerank. And each change to pagerank has been accompanied with much speculation in the SEO community about what the change was, with lots of graphs and experiments and gnashing of teeth. So no, I'm pretty sure we never have and never will see the source for PageRank.

Re:Nothing "magic" about it...

[CarrionBird]

You forgot the pigeons! And the "white pixel" collection.

Re:Nothing "magic" about it...

[geoffspear]

Please describe any search algorithm that could possibly have any reliance whatsoever on running on top of a linux kernel. Go ahead, try it.

Re:Nothing "magic" about it...

[Anonymous Coward]

Naturally, there isn't. However the parent poster may have had in mind the fiasco which transpired when Microsoft tried to turn its hotmail server into a windows operation.

Of course, SMTP has no reliance whatsoever on the linux kernel either.

Re:Nothing "magic" about it...

[magefile]

They did that once before. Used FreeBSD servers to run [H|N]otMail.

Re:Nothing "magic" about it...

[caluml]

I bet they have a Perl/C/C++/Java/PHP to .Net/ASP converter program hidden away somewhere.

Good for everyone

[meganthom]

With all the possibilities for type-specific search engines, this seems like a great step forward. Perhaps some day, I will be able to hum a song I heard on the radio into my computer and find links to the performer, composer, etc. If I am designing a new machine, I might be able to search for pre-existing components over large networks instead of through local libraries. Mmm...

Re:Good for everyone

[Lestat_79]

I think there is already a service like that http://name-this-tune.com/ Now everyone start whisteling "always look on the bright side of live, tedum, tedum, tedum ...

Can you say SEO?

[Supp0rtLinux]

Hmmm... I can only hope they release enough to better my understanding of their anti-SEO mechanisms. At least then I'll finally stand a chance of having a search for "Andrew Davis" actually show my personal website instead of page after page of that movie director dude (Independence Day, etc). :)

extending its API?

[eztiger]

Could this be an extension of the google API set ?

http://www.google.com/apis/

Co-inciding with a launch of gmail, this could lead to a slew of advanced 3rd party gmail apps like 'pop goes the gmail'

They Already Give Back

[N8F8]

Seriuosly folks, they probaly make me 30% more effective because I can find the information I need more quickly than anyplace else.

Re:They Already Give Back

[houghi]

they probaly make me 30% more effective

I hope they soon find a way to filter out sites that just put one or moreUsenetgroups on a website. The same goes for mailinglists.

When I am looking for something, I have to weed trough pages and pages of the same posting again and again. I start to use http://vivisimo.com more and more. The quality of the results is getting less and less.

Re:They Already Give Back

[Errtu76]

whenever i need to test a printer for printing colors, i just browse to google and hit the print button. it's alot faster than the standard windows testpage :D

and some more props: last friday afternoon there was a server problem. Found the solution on google. So thanks to google i was able to enjoy a warm meal :)

Click Thru

[marnargulus]

Click thru for when they ask for reg.
http://www.theage.com.au/articles/2004/06/21/10876 69904427.html?oneclick=true

Re:Is there anything Google can't do?

[Anonymous Coward]

*don's tinfoil hat*

You better give it back. Don gets very nervous without it.

I wouldn't.

[waynelorentz]

I know I won't be releasing any of the souce code to any of the back-end things I've written for my web site.

People would just laugh at it.

Disclaimer for the clueless: I'm not a programmer, so my code sucks so bad it's funny.

Re:I wouldn't.

[Fuzzums]

make our day, make us laugh.
c'mon and share your code ;)

Re:I wouldn't.

[Anonymous Coward]

make our day, make us laugh.
c'mon and share your code ;)

5 REM Look up document schemas some other time
10 PRINT "<HTML>"
20 PRINT "<HEAD>"
30 GOSUB 1000: REM Add meta tags
40 PRINT "</HEAD>"'"<BODY>"
50 GOSUB 2000: REM Add content
60 PRINT "</BODY>"
70 PRINT "</HTML>"
80 GOTO 3000: REM Post data

I wonder...

[PTDC]

...what license the code will be released under.

Re:Wrong

[bludstone]

You are just getting confused.

Google's service is to provide an audience for the advertisers.

Their customers are the advertisers, not us. We are just part of their MASSIVE (incredibly so) audience. We are part of what makes google sucessful. The fact we use it.

Thanks for giving back, google. Ill loan you my eyes a few more times :)

Re:Is there anything Google can't do?

[Directrix1]

I could be not very cool depending on if the patent situation associated with this "open code." If its heavily patented don't even read it, if you want real open systems.

Re:Is there anything Google can't do?

[Anonymous Coward]

It's working already. Of course Google still stores your every search. Google offers a service where they can correlate your emails (and your name) with all these searches. Google encourages you to never delete any of your mail from their servers. Google owns an almost complete archive of Usenet, even from times when people did not realize that their conversations would end up in a publically accessible archive. Google displays ads with javascript on countless third party sites, enabling them to track you on the web. The only thing that stands between them and one of the biggest commercially exploitable data-mine is their word. You see why they are desperate to maintain good public relations?

more

[subzero_ice]

Hope Google adds a site similar to akamai's NOC which displas google stats even though google downplays when it comes to numbers.

Finally...

[Metteyya]

...now Microsoft can start producing their own great and succesful search engine.

Re:Is there anything Google can't do?

[dnoyeb]

If Google releases the code they almost guarantee 1000s of new search engines popping up. Which guarantees none of them will ever be noticed.

Sounds like a smart decision. A lot cheaper than trying to sue competition into oblivion. Someone inside of Google is actually thinking.

Re:Is there anything Google can't do?

[Directrix1]

I -> It, ...on if the patent... -> ...on if there is a patent...

Giving back? Giving back to whom?

[Dagny Taggert]

Google provides an ad-supported service...for a profit. They are about to become a publicly-held company with the intention of...continuing to make a profit. I don't believe that Google owes the open source community anything---some may feel it's noble, but I think that it helps promote a certain "debtor" feeling, as if success should bring out timidity and apologies and giveaways. Google's competitors would be better served by innovating on their own and sticking it to them with better search results and more value-added features.

Google Toolbar on Linux

[Some guy named Chris]

Use Firefox, and get this.

Not a good move

[magefile]

Releasing the page rank formula would allow microsoft/yahoo/inktomi to create their own uber-search engines. But those companies have very different visions w.r.t. what makes a "good" feature. Google's tangible value is in Page Rank; while the sparsity of its pages makes it aesthetic and simple to use, that may not be enough to keep it alive.

OTOH, I would love to be able to play around with GMail's source code. And they could probably release code for, say, GoogleGroups, although I'm not sure why they'd want to or why I'd care.

Mistake IMHO

[PDHoss]

Google means well, but it almost certainly can't win with this move. They won't give the world the keys to the Google castle, of course, and the (often petty, let's face it folks) FOSS community will see what Google gave back and say "that's it?!?!" and be generally grumpy.

PDHoss

Here's some more code

[callipygian-showsyst]

using System.XML;

[...]

if (ContainsAdSenseAds())
{
pageRank++;
}

Re:Here's some more code

[Petronius]

using System.XML;

[...]

if (ContainsAdSenseAds())
{
pageRank++;
profit();
}

Re:Here's some more code

[BrandXandY]

If they released the code for the profit() function, the economy would probably collapse :/

Re:Wonder what it'll be!

[stanmann]

If it's patented, it has already been released... That is part of the patent process.

Re:what next? (gim for a name?)

[roror]

I for one am waiting for the brand new 'gim' tied to my gmail id. Heck, it's time for them to come up with MS passport replacement.

Re:Is there anything Google can't do?

[SilentChris]

"Is there anything Google can't do? Seriously, how cool is that?"

Just about as cool as when Apple, Sony, and even Microsoft (with restrictions) released some of their code. It's nothing new -- companies do it all the time for good PR and to help jumpstart initiatives. Question is, what code are they going to release, how much and when?

Time to "give something back?"

[Mr. Neutron]

In my mind, Google's free service has provided the world value somewhere in the order of trillions of dollars. Information gathering that once required a trip to the library and several hours searching through card catalogs and books, can now be done in mere seconds. I don't think they owe anyone anything.

It amazes me sometimes, their level of altruism.

Re:Time to "give something back?"

[value_added]

Since you brought it up, this article, Old Search Engine, the Library, Tries to Fit Into a Google World is definitely worth a look.

Re:Time to "give something back?"

[privaria]

What is it with this hostility toward people making a profit?

My local Catholic hospital once treated me for pneumonia by putting expensive antibiotics into my bloodstream for a few days. Without treatment, my odds of survival were about 50%.

Are they good because they save people's lives whether or not those people can pay (as I could) and regardless of their creed (I'm Lutheran, which is not exactly a religous alliance with Catholicism)? Or are they somehow evil because they charged for their services? I think the answer is obviously the former, but it seems a lot of slashdotters would opt for the latter.

Re:Time to "give something back?"

[mandalayx]

Trillons of dollars?

I hope that you, sir, stay far away from the IPO.

Re:Wonder what it'll be!

[magefile]

PageRank is well documented. It's not a neural net. I heard someone say that it's a 12 line perl script, but it's well-publicized that it's python. Might be 12 lines, though. 12 lines and a hell of a lot of data ...

No need

[Captain_Chaos]

Google doesn't have to give anything back, at least not to me. It's enough for me that they are by far the best search engine; lightning fast, accurate, comprehensive, free and with no obtrusive advertising...

Re:No need

[BenjyD]

No, of course not. But a stock of community good will can be a useful thing to have if times get tough.

I already have the code...

[Anonymous Coward]

... and all I did was visit they page and select "View Source" from the menu.

Re:Wonder what it'll be!

[haluness]

I have the algorithm but this comment does not have enough space for me to describe it :)

Re:Sure why not

[Gorgonzola]

Actually, I'd love to have a brain implant that would allow me to query Google just by thinking about it. It would probably disqualify me for any TV quiz show, but would otherwise be really nifty.

Quiet Period?

[Anonymous Coward]

This is a bit odd; I suspect they said more than they should have.

Google is in their pre-IPO quiet period; the SEC rules for this are fairly strict about forward-looking statements. I'm surprised this was mentioned at all.

It's a lot like slashcode

[Ieshan]

It's a lot like slashcode, really. Lots of great websites run slash, but none of them have the popularity or following that the original has.

Plus, Slashdot gets to "brand" websites with it's look, feel, and system. No matter how much you skin slashcode, it still feels like slashdot in the end, even if it wasn't intentional.

Read the slashdot FAQ, and you see exactly the same kind of response you would expect. Good luck making your own with just the source!

Re:It's a lot like slashcode

[LiquidCoooled]

My original thinking was to impliment the google algorythm inside mysql, so that any programmer can take advantage.

Instead of coding

select * from customers where notes like '*keyword1*' or notes like '*keyword2*';

the google function would search all relivent internal links.

select from customers where google('keyword1 keyword2')

It would have to be integrated into the engine itself, so that inserting data into the DB would be filtered through and scanned.

I don't care how the front end looks, and every project could benefit from using it.

This release of code to me seems a logical step, since a company can already purchase a google appliance which sits on your network and feeds personal results.

Re:It's a lot like slashcode

[Tony-A]

What makes Slashdot is its clientele, its commenters, readers and lurkers. The format, software and editors are important of course, which stories and when has an extreme effect on what is the current "in" subject. But I suspect there's many like me. A quick glance at the headlines and on to the comments.

Plus, Slashdot gets to "brand" websites with it's look, feel, and system.
No matter how much you skin slashcode, it still feels like slashdot in the end, even if it wasn't intentional.
The world-view and priorities affect many "minor" decisions in the coding. The net effect is pervasive and tends to dominate minor things like coding style and programming language. The effects survive multiple generations of hardware and software. Imagine the screams of agony if you could impose IBM's ERP systems on Sun or Sun's ERP systems on IBM. Imagine the disaster if you don't put your best people into implementing SAP. There's too much that is and has to be taken for granted that cannot be taken for granted.

from another response, "implementing the google algorithm inside mysql"
Hmmm, could be very interesting.
One things about "real" relationational databases has always annoyed me. What do you do when the existing available information is inconsistent? Sure, you can make everything consistent, but only by destroying information, likely critical existing information.

Re:It's a lot like slashcode

[burns210]

"It's a lot like slashcode, really."

YES! GiigkeFS has triple redundancy of all it's information, for recovery purposes, while slashdot reposts articles for... well, lets just call it 'recovery purposes' too.

"No matter how much you skin slashcode, it still feels like slashdot in the end, even if it wasn't intentional."

But a textfield and output is so generic, you don't have the /. sidebar and color scheme(even with themes and whatnot) that just scream ripoff.

Re:Is there anything Google can't do?

[WormholeFiend]

That or enabling private companies or other big organizations which do not want an internet access to their intranet, to offer a google-like search engine to their users.

I've seen some intranet search engines and believe me, they are teh sux0r compared to google.

Not the first time.

[TXP]

They released some of their code in previous programming contests. The code allowed users to access their compressed data file format(compressed/indexed HTML) and quickly run seachs on them. As well they provided 20->200 megs of sample data. (Something like that) It was a couple years ago. April 30 2002. http://www.google.com/programming-contest/

"Google plans to reveal some of the code"

[SageMadHatter]

The key word been some. Expect the following precious gem to be released:

int main()

=P

Re:Is there anything Google can't do?

[arkanes]

Google makes alot of money from providing this service to companies already. See the ever-so-cool Google Search Appliance. I don't know why they'd cannibalize that market.

Re:Is there anything Google can't do?

[Dirk Pitt]

what code are they going to release

They're releasing all the code that changes the bannerhead to some cute picture on holidays. Powerful stuff.

no need for gmessenger

[muyuubyou]

When we have Jabber. Jabber is as open as a messenger can ever get.

Blockquoth their site:

the Jabber protocols are free, open, public, and easily understandable; in addition, multiple implementations exist for clients, servers, components, and code libraries.

You can run your own server. Jabber is never down, is and will always be open, and doesn't crap over you with ads.

Re:no need for gmessenger

[outZider]

And just like many other open source apps, doesn't really outdo anyone, so no one seems to care. ;)

Re:Giving back? Giving back to whom?

[tukkayoot]

Well, from what I've read, part of the reason Google is so successful is because they've had a lot of smart people contribute good ideas that have allowed them to thrive. Part of the way they attract and keep these smart people is to let them entertain and challenge themselves by working on personal projects, do research, ect. and release the results to the general public.

This just sounds like it's an extension of that philosophy. Some of the smart people who have worked on Google code know their code probably isn't necessarily going to do them damage in the hands of their competitors, and would like for that code to be shared with the world, for, I guess what you would call their "academic merits".

So it seems to me that Google is going on this recruitment tour, demonstrating how, unlike many companies, by working at Google, your labors aren't all going to be locked in a secret vault never to be shared with the world. Knowing that this appeals to some smart/talented people, Google's "charity" seems rather pragmatic.

Devil's Advocate again....

[jwcorder]

I hate to be the closed minded one again, but I think this is a bad idea. One of the coolest things about google is that no one really knows how it all works. It's like Willie Wonka the candy just shows up in a little box. And with the competition revamping and reworking everything they can to catch up to the googlites, I think this is a bad idea.

I think that 5 years from now, they may look back and say, you know if we hadn't reveal line 5267 of our code, MSN and Yahoo wouldn't be kicking our asses right now.

Of course my opinion is worth just about as much as the paper you could write it on.

Re:Is there anything Google can't do?

[Anonymous Coward]

A patent has nothing to do with code or implementations. If it's patented, you would not be allowed to implement it even if you had spent your entire life in the cellars of a 300-year-old monastery, receiving only food and water from the vow-of-silence monks, and seeing, reading and hearing nothing about the outside world whatsoever.

Copyright is a different matter, but with patents, you're screwed either way.

Bork bork bork!

[Peldor]

All I want for Christmas is their Swedish Chef translator.

Re:Bork bork bork!

[fanfriggintastic]

Seriously, bork bork is hilarious. Via Opera Bork Bork Edition, it vastly improved the MSN home page!

Gmail tech ?

[dabossuk]

Hi

Does anyone know if gmail was an inhouse devleopment ? Or is it third party. I am interested on becuase I notice that when it load of gmail it says "please wait ..." can I assume that in fact this is some sort of Java applet that is loading. I noticed that someone sent me an email to my account - and it appeared without me having to rfresh the browser - did not see a hidden frame / meta fresh either - interested to know how they are doing this.

Paul

Re:Gmail tech ?

[Richard_at_work]

Its almost entirely javascript with a hell of a lot of preloading going on. Its a very good system actually, Im amazed that everything works identically in both IE and Firefox.

Re:Gmail tech ?

[TheWickedKingJeremy]

Also - and I'm not sure that Gmail uses this - but you can hit a database and update a page dynamically via javascript without refreshing the page or using a hidden frame. This is accomplished by embedding an XML component into the page, and using this component, you can talk to a database and handle the results all in javascript. The syntax is different for IE and Firefox (W3C) but it works in both.

Re:Gmail tech ?

[dabossuk]

Hi Have you any idea how this could be done in ASP.NET and or ColdFusion ... Paul

Re:Gmail tech ?

[TheWickedKingJeremy]

Well, what I was talking about is independant of any server-side code like ASP of CF. Essentially, you embed an XML component into a web page, so it is sitting on the client machine. Using javascript, you can manipulate this component to, for example, go fetch a page on some server. That page can be a server-side page (like an ASP page) which means, it can hit a database, formulate some result XML, and return it. The XML component on the client now has this XML document loaded, and you can parse it and do with it what you will.

So essentially, a sitting HTML page can dynamically talk to a database and display results without ever refreshing using this method. The real work is being accomplished by the XML component, but that is invisible to the user (unless they are monitoring their HTTP traffic).

Again, I don't know that gmail uses this technique, but it certainly could since it requires javascript anyways.

Re:Gmail tech ?

[dabossuk]

Hi Ah got you ... do you know of any examples I could take a look at that uses this trick ? Thanks Paul

Re:Gmail tech ?

[christopherfinke]

Gmail refreshes every two minutes or so to check for new mail. If you have it open in a tab in Mozilla, you can see it change from "Gmail - Inbox" to "Loading..." and back to "Gmail - Inbox".

Re:Gmail tech ?

[TheWickedKingJeremy]

Ok, I made up a sample page... basically it hits the server (a .ASP page) with a request (in this case, a fake little "getUserInfo" request) and gets back some XML with information about that user. It then displays that information on the page, without having reloaded the page or used a hidden frame. I commented my javascript code, so if you view source you can see what it is doing.

The cross-browser wrapper to work with these XML components in IE/Firefox is something I put together for a project at work to simplify things - I can't say its 100% perfect or complete yet so use at your own risk.

here is the demo page:

http://www.nextradiosolutions.com/shared/slashTest /myPage.html

here is the xml-wrapper for IE/Firefox that the demo page uses:

http://www.nextradiosolutions.com/shared/slashTest /domXML.js

Re:Gmail tech ?

[TheWickedKingJeremy]

Oops, there is a space in those links... here are some direct links:

The demo page

dom-xml wrapper

Re:Gmail tech ?

[sahala]

Does anyone know if gmail was an inhouse devleopment ? Or is it third party. I am interested on becuase I notice that when it load of gmail it says "please wait ..." can I assume that in fact this is some sort of Java applet that is loading. I noticed that someone sent me an email to my account - and it appeared without me having to rfresh the browser - did not see a hidden frame / meta fresh either - interested to know how they are doing this.

Not sure about whether it's in-house or not. There is no Java applet involved, however for IE an ActiveX object ('Microsoft.XMLHTTP') is loaded to send and receive XML messages. Everything else is straight DHTML (Javascript + DOM). It works on a wide range of browsers because they kept the design simple. There could be some browser detection logic to handle specific browser nuances.

There are indeed frames involved. The topmost document contains a frameset containing 2 frames, each of which contain necessary IFRAMEs. I believe the "no-refresh" feature is done through XMLHTTP, although I could be wrong.

Re:Gmail tech ?

[Fuzzy_Nuts]

Still haven't gotten to see Gmail in action wouldn't mind getting to test the service myself... could someone send me an invite so i can check it out?

The Real Prize

[Skip Head]

I'm hoping they release the source code to their web page. Analysis should provide some insight into how they produce their easy-to-understand user interface.

I know!!!

[Anonymous Coward]

Maybe they could give us the source code to their webpage. It would be cool to find out what HTML code they used to get it to look like that

Visibility

[4lex]

As make extensive use of linux, might I suggest that they include a tiny picture with the text "Google is powered by linux" with the search results? Maybe at the bottom, where you can't barely see it... but it will be enough to raise linux visibility a lot, if google users (i.e.: every internet user) realizes that linux is powerful enough for google.

I agree

[Per Abrahamsen]

I suspect almost all free software developers use services provided by Google. I certainly feel Google in so many ways have made the net a better place for me, that I feel more than compensated for any software of mine they may use.

It might be a good idea and in their own interest for them to contribute financially to some of the free "core" technologies they use. I don't see them in the osdl memberlist for example.

Page rank no magic -- just plain old science

[ikoflexer]

Seeing some of the euphoric comments here is pretty interesting. It just shows how good googleans are at maintaining their "we're the good guys" image. And that's good, because indeed they seem to have cleaner business practices than many other.

However, let's set the record straight here. There is no magic algorithms and most likely no new science in the technology google uses for search. The original page ranking paper published by Sergei et al. explained the entire algorithm, and data structures they needed to have a fast search. The paper was quite detailed. It's good old computer science. Nothing extraordinarily outrageous about it (from a mathematical standpoint), except that it worked damn well: it leveraged human knowledge, which is something a lot of people don't yet either understand the importance of, or don't know how to do it.

Since then, google has improved on the original algorithm, data structures, and overall implementation. But the main technology was in plain sight from the beginning.

My guess to why google is releasing some source code is because it will pay off for them. Forget the "give back" bit. It'll improve the image, and, potentially they might benefit from people using that source. How? Not clear yet until we see the source code.

In any case, as slashdotters ubergeeks we should keep a cool head and look at what google will show. I'm 100% sure that if they said they release some source they'll do it. You see, it would hurt their image if they didn't, and they're too good at image management to make such a stupid mistake.

It's obviously a strategic move on their part, just that we don't see yet the real reason for it.

BTW, it's just a matter of time for MS or Yahoo to incorporate this type of technology into their search engines, and google knows that. As they loose their technological edge over the competitors, keeping a good image is very important. As is branching out into other services (e.g. gmail).

Just my 2c.

ummm... no

[Ender Ryan]

I'm sure Google will release code that is interesting, ie. show how their search engine works on a technology level, but I doubt they will release code that would enable someone else to build another Google-quality engine without doing their own work. For example, their pageranking algorithms are mostly understood, but I doubt they will reveal how to finely tune the process as Google has done.

Of course, I could be wrong, I couldn't even RTFA...

It'll just be PageRank

[Lewisham]

Call me cynical, but I highly doubt Google are going to release anything useful. Maybe an update to the API.

What I'm thinking is they'll make a big fanfare about PageRank, and release the same thesis paper Brin and Page wrote at Stanford again, with a couple of tweaks.

No more.

google sets

[hyperstation]

i'd like to see how google sets works, myself...

Re:google sets

[bairy]

Same way as a thesaurus I expect. Set of keywords bunched together that all appear when one is called.. depending on context of word of course

Re:Is there anything Google can't do?

[Jim_Maryland]

My project was considering the use of the Google Search Appliance to keep in line with one of our main customers who already uses one. In our situation though, we wanted to extend the capabilities of "Google" to include geospatial searching. This was late in 2003 so maybe things are changing a bit, but back then, Google wouldn't allow you to do anything to their appliance (basically a black box with a minimal customization interface). They weren't even considering non-disclosure agreements (NDA) with us (one of the larger companies in the world). I'm sure a couple people I work with will be following the possible opening of Google.

API and source code: This could get interesting

[WallaceSz]

Google released its Web APIs back in April 2002. Since then a number of successful products have evolved, including Google Dance Tool and the Google Alert alerting service.

Google has been unclear about their future plans for the API, though it seems that it has allowed some apps to go commercial - see this article.

One wonders if this source code will actually add value to the developer community, or is simply a way of counteracting the capitalistic vibes of the IPO.

Re:API and source code: This could get interesting

[markkellman]

Indeed. I'm going to go with PR stunt until they prove otherwise.

Re:Mod parent down!

[callipygian-showsyst]

What did you want to see? Objective-C code? Java (a proprietary language?) code? I can't figure out why the /. crowd is so NUTS!

In the near future...

[saha]

Microsoft grabs the code, develops on top of it, doesn't reveal any of their additional work, then tries to patent and copyright the code.

Microsoft says Web site violates copyright

Time based hardware button for application launch i.e. Double click

I think its this bit :

[The_great_orgazmo]

Google

HetInternet Afbeeldingen Discussiegroepen Gids

document.f.q.focus();

Geavanceerd zoeken
Voorkeuren
TaalhulpmiddelenHet web doorzoekenZoeken in pagina's in het Nederlands

Advertentie programma's - Alles over Google - Google.com in English //Maak van Google je Startpagina!");} //-->

©2004 Google - Zoekt in 4,285,199,774 webpagina's

I have the strangest feeling...

[Shoten]

This is so unlike any reaction I thought I might have to such an announcement, but I really do feel that Google, by giving us such a great search engine (and working so constantly to improve it) is giving us enough. I wouldn't want them to risk any of that by giving away bits of PageRank!

Re:Is there anything Google can't do?

[Anonymous Coward]

Google owns an almost complete archive of Usenet, even from times when people did not realize that their conversations would end up in a publically accessible archive.

If the conversations were posted to a publically accessible forum, what's the big deal about a publically accessible archive?

Re:Sure why not

[tarunthegreat2]

Well I dunno about the guy above, but I was planning on get a mobile phone implanted in my head with an interface to my brain...I thought it was a great idea until those fscking telemarketers started calling....And boy did I get accidentally dial a bunch of wrong numbers!

Some more code released....

[rollthelosindice]

use strict; # pragma

only on slashdot

[Lehk228]

would you get criticised for bugs and deviation from standards in your joke source code.

Not code, just papers

[glinden]

In the article, Wayne Rosing explicitly says that Google is not planning on open-sourcing the Google code base, but that they will publish academic papers on their work. "I'm not saying we're going to open-source Google, because that would be a little dumb when we have these Microsoft guys making noise. . . We're encouraging the software engineers to submit papers where it makes sense, particularly where it is landmark work and it is really important that other people know."

Google already has published a number of papers on their systems, including descriptions of PageRank, their clustering architecture, and their high availability file system (the Google File System). Seems like this is merely an announcement that they intend to do more of the same.

Re:Not code, just papers

[Kupek]

What he meant was they're not releasing all of it, just some of it.

"There have been a lot of conversations in the company in the past two months about (how) . . . it's time for us to give something back. So our technical director, Craig Silverstein, has started a project to look at all the Google code and start figuring out what parts of it we want to give back," Rosing said.

Re:Not code, just papers

[0x0d0a]

Note that this is very unusual for the non-academic tech researchy community, par for the course for Google, and just makes us all love Google more.

Sneak Preview of the Code from Google

[GweeDo]

/*
* Allocate 'size' units from the given map.
* Return the base of the allocated space.
* In a map, the addresses are increasing and the
* list is terminated by a 0 size.
* Algorithm is first-fit.
*/

ulong_t
atealloc(
struct map *mp,
size_t size)
{
register unsigned int a;
register struct map *bp;
register unsigned long s;

SCO's not going to be happy about this ;)

Giveth andth taketh

[Mulletproof]

"here have been a lot of conversations in the company in the past two months about (how) . . . it's time for us to give something back. ...Which would imply they've taken something from everybody??? Yeah, that just gives me all kinds of warm fuzzies about gMail and their other future plans.

How to give back and make it count.

[zenmojodaddy]

If they really want to give back, they should use Google to push Firefox and Opera. They have the clout to help break Microsoft's chokehold on the net before Longhorn finally arrives... if they decide to put their closed-source money where their open-source mouth is.

Re:Is there anything Google can't do?

[Anonymous Coward]

There are things that you have no problem saying in front of an audience of a few hundred people but would not dare say in front of your employer or a customer. The consequence of "it was public, so a public archive makes no difference" is that people constantly have to act like they're interviewing for a job or selling something to a customer. People need a mode of conversation which is not logged, and Usenet did offer something which was reasonably expected to be transient.

Re:Sure why not

[88NoSoup4U88]

And result and endless stream of pr0n results :) Flooding your computer in a self-created DOS ;) (denail of sex ?) hhmmm

Not buying it (Re:Excellent!)

[otisg]

Are people really buying this??
I am usually optimistic, but I find this hard to believe. Google is in the pre-IPO mode, #1 search company with Microsoft on its heals, and so on. Who in the right mind would provide information about their business, let alone anything concrete? If you read interviews with Google's C*O people, you will not that they always avoid answering questions like "How many servers does Google really have? 10K, 100K, or more?". Nobody has ever answered that simple and seemingly benign question, so why would they now reveal what makes them number one?
My guess would be that this is another elegant marketing trick playing the good guy (e.g. no ad banner, no sponsored links (I'm talking early Google), no evil software campaign, then the generous 1GB mail, etc.) in every possible geek (read: early adopters/thought leaders) facet.

Re:Not buying it (Re:Excellent!)

[Tony-A]

Who in the right mind would provide information about their business, let alone anything concrete?

Google is in the information business. They understand information and what conclusions can and will be inferred. Their C*O people don't blab to satisfy someone's idle curiosity about seemingly benign questions. As a guess, Google has, does, and will continue to reveal what makes them number one. But they do so on their own timing. You've listed what's probably the main one, "no evil software". That's just a public statement of existing policy.

My guess would be that this is another elegant marketing trick playing the good guy
And if they are the good guy, ...
Actually methinks they are in a position where they can be and it is very much in their long term interests to actually be the good guy.

Re:Is there anything Google can't do?

[j1m 5n0w]

In my opinion, Google's market dominance has more to do with massive computing and network resources than technology. (Their reputation of being not evil doesn't hurt, either.)

Their architecture is relatively well-understood. If you want to know how google works, read The Anatomy of a Large-Scale Hypertextual Web Search Engine, The PageRank Citation Ranking: Bringing Order to the Web, and some of the other papers from Google. (They probably have some special tricks they haven't revealed to the public, but the basic algorithms aren't that complicated.)

It's hard for anyone to compete with Google without having enough bandwidth to crawl a large portion of the web on a regular basis, and then store the resulting data structures in RAM.

-jim

Re:Sure why not

[Gorgonzola]

My brain is perfectly capable of generating high resolution material of the nature you are referring to, thank you. I was more thinking along the lines of being able to look up things on the fly. It would be as if a significant part of our collective knowledge would be available by merely thinking up the right questions.

Google's top asset: solid reputation

[Espo_SHIZ]

Google's been on top for too long. Microsoft and other search engines are really starting to push competition. But even if they equal or surpass Google's services, they still won't succeed (lawfully, at least). Google has something none of them have, and something most of them can never get with their histories: Trust.

A little pressure goes a long way

[cybrthng]

Being the owner of Mozdex.com i was proud to find out that my name gets tossed around the Google campus and it appears some of the ideas of my project as well as the developers of the Nutch, Lucene and Egothore projects have been heard.

I'm not to sure what google would actually release, however it is interesting they're trying to back pedal on some of there policies and recent patents they had been filling

I don't wear a tinfoil hat, but i don't like the enormity and control google is pushing over the search engine world as well as the advertising/contextual marketing world. They have some of the most assinine Terms of Services and agreement policies i have ever seen. basically if you choose google you have to walk on pins and needles because you don't (nor never will) know how they operate and nor do you know if they will change something that will lead you to loosig all of your revenue or being banned, kicked or whatever they choose.

See what it is all about.

Google doesn't quite own PageRank

[Jayfar]

The original pagerank patent 6,285,999 lists larry page as inventor, but The Board of Trustees of the Leland Stanford Junior University is the assignee. Google has an exclusive license on that patent through 2011. There's a later patent, 6,526,440, listing Krishna Bharat as inventor, Google as the assignee. The latter patent appears to be a minor refinement, per the abstract:

"A search engine for searching a corpus improves the relevancy of the results by refining a standard relevancy score based on the interconnectivity of the initially returned set of documents. The search engine obtains an initial set of relevant documents by matching a user's search terms to an index of a corpus. A re-ranking component in the search engine then refines the initially returned document rankings so that documents that are frequently cited in the initial set of relevant documents are preferred over documents that are less frequently cited within the initial set."

this article is redundant...

[Milton Waddams]

...their source code is already available! just point your browser at google.com, left click on the page and choose "View Source"

Ahh, some more...

[The Ribena Kid]

...code that SCO can now claim they own the rights to. Do you think I should buy SCO stock now before investors realise SCO is talking even more sh*t for their pump and dump scheme? ;-)

Google is NOT that great. Just above Average.

[DRWHOISME]

Every article i try to find now has been google bombed. It's harder and harder to find anything on Google.

I don't know or why people assume that their low tech algorithmn is so great. There's nothing that prevents crass commercial companies from telling their customers or paid cronies to link to them.

I think Google is very overrated and can't handle searches on a variety of topics and I can't believe the media attention that this lone search engine gets when there are others just like it.

Google is no better than a TV network

[MushMouth]

Hear Hear, this google worship amazes me. I think I have enjoyed far more free as in beer content from ABC/NBC/CBS than I have from google, yet you would never hear these guys adulated in the same way that google is. Yeah they use linux but so do plenty of other companies, unlike google they don't find it necessary to say aren't we so effin cool, we use linux.

Re:Google is no better than a TV network

[timpaton]

Yeah they use linux but so do plenty of other companies, unlike google they don't find it necessary to say aren't we so effin cool, we use linux

You mean "other companies" like Microsoft? ;-)

You're right - they tend not to be so vocal about their use of Linux servers...

Re:Warning: parent is a terrible programmer

[ubern00b]

Well, AFAIK, the output stream flushes itself upon destruction (if there were multiple lines with "cout", then he would have to flush between them), and main() will return 0 invariably (Google always works, right? It doesn't need to return error codes.) Furthermore, this is a joke. Who would actually bother to pay attention to every detail, when this post will eventually sink into obscurity anyway?

Slightly Offtopic, but ...

[unnique]

We might be enjoying the benefits of Google, but are Google's employees themselves not content? Acoording to this link on NYT[reg. required, yada, yada], the top search for "out of touch management" in Google caused the search engine to list as its first result a page describing the company's top management.

Give back platform access, not (only) code ...

[__aadkms7016]

Google should consider "giving back" by providing a platform for services to run on -- a PlanetLab for the SourceForge world. Google code isn't about a single server, it's about clusters of servers, and few people have the budget and experience to set up distributed clusters on their own.

Hire me!

[leeet]

I don't have a PhD but I have ideas. And it would be giving back to the society too.

What about Ask Jeeves?

[Tiffsterr]

All this talk about Google makes me want to barf. Google is so boring! What about Ask Jeeves? This small innovative search engine company is roaring and we are hiring Software Engineers, Research Engineers, Computational Linguistics & NLP Engineers, Product Managers and more in New York, New Jersey & Northern California. Please check out our jobs at www.Ask.com to see what is new. I currently have an urgent position for a Software Engineer, Content Group. Here is the link to the job description. Regular full-time & very competitive compensation. http://www.askjeevescareers.com/askjeeves/NC.asp?I D=234 Thank you! Tiffany @ Ask Jeeves, Inc.

Alternatives to GoogleFS?

[leeet]

Aren't there some open source alternatives to the GoogleFS? Are they "production stable" ? Would we benefit from Google releasing some of that code?

Bill Gates hopes it's under the BSD license

[Anonymous Coward]

They've been waiting for this. Can you imagine? Just grab Google's code, implement it in the new msn search and don't tell anyone...

The math of PageRank...

[dev2718]

...isn't really much of a secret. In fact, it's beautiful because it's about the only way to solve that problem given the particular constraints (and something anyone who's taken linear algebra will immediately understand). Google's particular implementation of that algorithm isn't any more useful than, say, RSA's particular implementation of public key encryption. I would much prefer they release something infrastructural (eg, the distributed filesystem) rather than algorithmic.

"A Note on the Eigenvalues of the Google Matrix"
http://arxiv.org/abs/math.RA/0401177

Interview with Matt Wells (GigaBlast)
http://www.acmqueue.com/modules.php?name=Content&p a=showpage&pid=135&page=1

Re:Sure why not

[Mazzie]

Actually that would be a pretty cool gameshow. Ask people incredibly difficult questions (history, literature, sports, etc) and let them use Google to find the answer. Viewers at home could play along and try to find the answer faster than the contestants.

Giving back or snapping back?

[Mr_Huber]

After reading through this article about the fall of the Win32 API, I think Google may be doing something a bit more clever than simply 'giving back'.

The article discusses how the plethora of APIs Microsoft is shipping and uncertainty of just when the APIs will be on real hardware have caused new development to move to the web (for example, building a new email client). It also mentions there are several drawbacks to web programming, but the author expects them to be overcome soon, further sapping the appeal of the Win32 API.

Perhaps this is exactly Google's strategy. How better to further enhance the desireability of web programming versus desktop programming than by demonstraiting in source how to perform some really compelling features? How many developers will take these techniques and use them at the heart of new killer apps; apps that run on any web browser rather than locked to a specific API?

Google may be giving a small bit away, but the potential gains in mindshare among developers may be much, much greater than the loss of this already paid for source code. Further, if the code in question relies on a particular technology, protocol or standard they are well versed in, they have succeeded in enticing developers to play in their sandbox.

Quite clever, really.

Won't be substantial...

[tarp]

They won't release any really "vital" pieces of code, because it would reduce their competitiveness, and the most intelligent of the porn spammers and viagara salesmen would find weaknesses in the search /indexing algorithm and exploit them to bring their pages higher in the rankings.

Now if only they'd hurry up and open up gmail to the general public! I've had no luck in getting an invite :-(

Re:Is there anything Google can't do?

[42forty-two42]

Is using the X-No-Archive header so hard? I belive they have a form to remove your messages after the fact as well.

Re:HTML "source code"

[trippinonbsd]

Their site has to be as compatible as possible. That is probably why they cant be w3c compliant.

In that case...

[MrNemesis]

...how long before the search code is integrated into SETI@home and discover aliens searching with their 802.11xxx for hot alien pr0n?!

(It's a joke. Larf.)

+1 Funny

[trippinonbsd]

Ask Jeeves is lame.

Re:+1 Funny

[Tiffsterr]

You are hurting me trippin... Tiffany

SCO

[xp]

What if SCO finds the source for grep in google.c?

----
Is Your Boss A Muppet?

Re:Giving back? Giving back to whom?

[The Raven]

I read your sig before I read the comment, and wondered if it was related to Ayn Rand... then read the command, and was pretty sure of it... then the email address, and was embarrassingly certain. :-)

Rock on with your Rational self.

Partly True

[overworked underpaid]

I was at the presentation in Melbourne. Wayne made it very clear that they did not intend to open source the Google search engine. That would be commercial suicide (my words, not his).

What he did suggest is that they are keen to give something back to the OSS community that has given them so much. The impression that I got was that they would return much of their operating systems work to the community. They do spend a great deal of effort in Linux kernel hacking and so forth.

I was very pleased to hear this from a senior member of Google's engineering team. We're yet to see the fruits of this decision, but the policy change has to come first. Let's hope it works out.

I got ahold of it!

[acidrain69]

Exclusive code-leak to slashdot!

void main()
{
webcrawler();
}

the end!

Re:I got ahold of it!

[takkaria]

int main(void)
{
webcrawler();
return 1;
}

Open-sourcing Google code: now or never

[j.leidner]

If they indeed intent to open-source some code under one license or another, they need to do it before they go public, as I suspect it's impossible to convince a post-IPO company's board to give out source code, or any other asset for free.

Well...

[System.out.println()]

IMO AdWords alone is enough to seperate Google from any TV network. Non-intrusive, non-annoying, non-bandwidth hogging ads that are actually relevant to page content, as opposed to taking 15-20 minutes of every hour to shove random products down my throat.

I'm not sure where you've heard Google claim to be cool because they use linux. Maybe something about them being cool because they have a custom variant of linux.

Also, you don't hear the networks elevated because you don't communicate with people over your TV. People on the net (who are likely to dislike TV and prefer the net) are the ones you'll hear, and that kind of person is likely to use and love google's services often.

Come to think of it, why are you here? :)

4 invitations left

[KrisCowboy]

I got 4 gmail invitations left from my second quota. Any takers?