Google Programming Contest Winner

asqui writes "The First Annual Google Programming Contest, announced about 4 months ago has ended. The winner is Daniel Egnor, a former Microsoft employee. His project converted addresses found in documents to latitude-longitude coordinates and built a two-dimensional index of these coordinates, thereby allowing you to limit your query to a certain radius from a geographical location. Good for difficult questions like "Where is the nearest all-night pizza place that will deliver at this hour?". Unfortunately there is no mention whether this technology is on its way to the google labs yet. There are also details of 5 other excellent project submissions that didn't quite make it."

Comments

if i'd only known

[oogoody]

they wanted such boring submissions.
The winning idea was cool, but the rest looks
like free development for google rather
than something novel.

Re:if i'd only known

[Indras]

like free development for google

Let me quote from the homepage of the annual contest:

"Grand Prize

$10,000 in cash

VIP visit to Google Inc. in Mountain View, California

Potentially run your prize-winning code on Google's multi-billion document repository (circumstances permitting)"

Re:if i'd only known

[Peyna]

IIRC they also mentioned something about the potential offer of employment, then again, maybe I was just imagining that.

Re:if i'd only known

[Peyna]

If my submission won, I would apply for a job =]

Re:if i'd only known

[Indras]

I would ask for royalties! A buck for every time someone viewed a page containing my code :o).

I see one being implemented soon

[Masem]

From the hon. mentions:

Laird Breyer, for his project, Markovian Page Ranking Distributions: Some Theory and Simulations. This project examined various properties of the Markovian process behind Google's PageRank algorithm, and suggested some modifications to take into account the "age" of each link to reduce Pagerank's tendency to bias against newly-created pages.

This may help to defeat the current practice of overloading the PageRank results of a given key word as to point to a given page by having people link to that page with a link containing that keyword, aka "Googlebombing". I do think that the winner is a very interesting and useful project, this latter one will probably be implemented ASAP.

Re:I see one being implemented soon

[3.5 stripes]

Yeah, it's a pity things like this are less exciting, it's a much needed addition, just not as "innovative".

Re:I see one being implemented soon

[Anonymous Coward]

Try the Google Glossary to find definitions of words or phrases.

Markovian Dependece- The condition where observations in a time series are dependent on previous observations in the near term. Markovian dependence dies quickly, while long-memory effects like Hurst dependence, decay over very long time periods.

I sent something into the contest.

[thedanceman]

But I guess they thought there was no need for -thedanceman- on the google site.

Re:I sent something into the contest.

[xtremex]

Grrrrr! I can't play WMV files in Linux! Anyway anyone can convert them to mpegs or somethign?

What a great idea

[TedCheshireAcad]

If only more pizza restaurants in my area had web sites. Soon enough, I won't even have to pick up the phone to make my food come to me! I wonder if the delivery guy will bring the pizza up to me at my computer. Hmm...

Google Search

[iramkumar]

Search => Osama Bin Laden
Latitude/Longitude => 37/180, Pak
Capture ...

If this would have come out before we could have saved a country ...

Idea for a Google Query.....

[ReelOddeeo]

Where is the nearest server in my jurisdiction where I can download....

• MP3's
• Warez
• Pr0n
• Explosives making instructions

And worst of all....

• DeCSS

We've got to stop all of the terrorists in the categories mentioned above!

"Google Sets"

[ralian]

It really seems to me like the "Google Sets" feature recently made available at Google Labs is an implementation of Zhenlei Cai's submission(although the details are extremely sketchy in the Google announcement). If this is true, I wonder why they couldn't implement the winning idea too?

more details

[Alien54]

Daniel's project adds the ability to search for web pages within a particular geographic locale to traditional keyword searching. To accomplish this, Daniel converted street addresses found within a large corpus of documents to latitude-longitude-based coordinates using the freely available TIGER and FIPS data sources, and built a two-dimensional index of these coordinates. Daniel's system provides an interface that allows the user to augment a keyword search with the ability to restrict matches to within a certain radius of a specified address (useful for queries that are difficult to answer using just keyword searching, such as "find me all bookstores near my house"). We selected Daniel's project because it combined an interesting and useful idea with a clean and robust implementation.

This is impressive bit of database manipulation. Somehow I didn't think that all of the datatypes, etc would be so easily parsed.

Although I do recall telephone directories that used to give you results for a specified radius for certain types of businesses

Re:more details

[Matey-O]

There's code available now that does this for zipcodes. see http://www.zipmath.com/ (And using Mapquest as the black box, street addresses too.) Tieing it into google is a nifty bit o kit tho'.

Re:more details

[Lars T.]

Sounds like this improvement isn't much use outside the US.

Re:more details

[nathanm]

Sounds like this improvement isn't much use outside the US.

His current implementation wouldn't be of much use outside the US, but the code could be used with non-US data elsewhere. The TIGER & FIPS data is just geographical & address information commonly used in GIS. I know the UK has similar data available, other countries probably do too.

Google chose his project because his code was clean and robust. It shouldn't be to difficult to get it to work with other data.

Re:more details

[Chester K]

This is impressive bit of database manipulation. Somehow I didn't think that all of the datatypes, etc would be so easily parsed.

Although I do recall telephone directories that used to give you results for a specified radius for certain types of businesses

That's just a standard spatial query. It's easy to implement an R-Tree to be able to do (relatively) quick "give me points within x meters of this one" type of searches on a database. There's nothing extremely revolutionary about Daniel's project, anyone with some basic geometry knowledge and the patience to download the 33GB of TIGER data could have done it within the course of a few weeks. (Ironically enough I've been doing the same thing with 1.2 million addresses against TIGER data for the past month.)

But that's the true genius and beauty of it. Now that it's been said, it's such a mindbogglingly obvious and useful application of web search and spatial search technology that it's hard to believe nobody thought of it before.

I'd be honestly surprised if Google doesn't run with the ball and fold it into their main search engine. The only thing standing in the way is the storage space and CPU time to do it.

Re:more details

[xant]

I'd be honestly surprised if Google doesn't run with the ball and fold it into their main search engine. The only thing standing in the way is the storage space and CPU time to do it.

. . . Two things Google should have no trouble finding.

404 Page Not Found ?

[bigmouth_strikes]

I'm surprised that there are so many 404 Page Not Found errors in Google's search results, even on the top hits.

Shouldn't Google automatically check results that a user follows and flag those that cannot be displayed ?

Re:404 Page Not Found ?

[DrSkwid]

#

Thomas Phelps and Robert Wilensky, for their project, Robust Hyperlinks. Traditional hyperlinks are very brittle, in that they are useless if the page later moves to a different URL. This project improves upon traditional hyperlinks by creating a signature of the target page, selecting a set of very rare words that uniquely identify the page, and relying on a search engine query for those rare words to find the page in the future. For example, the Google programming contest can be found using this link.

Re:404 Page Not Found ?

[rmohr02]

Yea, I've realized that, but then you realize that Google caches most of the web and nearly all of the links produced in search results. So if you get a 404 error you go back and click on the cache link.

Re:404 Page Not Found ?

[PhilHibbs]

Google's links are not redirected via their server, and a lot of people would object to them "gathering data on their users' browsing activities". However, automatically checking the top link after each search (or scheduling it for checking) should be possible.

What should they do if a page is unavailable, though? What if it's only down for a few seconds?

Re:404 Page Not Found ?

[LinuxHam]

Shouldn't Google automatically check results

I would much prefer to see them improve the ease of browsing their cache. Specifically, if a cached site is 404, then present a cached version of the site where all clicks within the site simply link to the cached version, unlike today where all clicks are native (and therefore lead to more 404's). Granted that wouldn't be of any use for links to dynamic pages, but anything is better than what they have today.

Re:404 Page Not Found ?

[General Wesc]

And of course there's the Mozilla Google Toolbar for people who don't use IE.

Re:404 Page Not Found ?

[srvivn21]

Interestingly enough, the google translation page does this. Translate a page, and any links on that page lead to translated pages. Very convenient.

Re:good news for open source?

[rmohr02]

It is great to see open source software such as google

Google's not open source. They support the open source community, but they don't release the code the their indexer. This is all the information that they give out about their code:

PageRank Explained

PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important."

Important, high-quality sites receive a higher PageRank, which Google remembers each time it conducts a search. Of course, important pages mean nothing to you if they don't match your query. So, Google combines PageRank with sophisticated text-matching techniques to find pages that are both important and relevant to your search. Google goes far beyond the number of times a term appears on a page and examines all aspects of the page's content (and the content of the pages linking to it) to determine if it's a good match for your query.

Re:About the "Former Microsoft Employee" bit..

[GT_Alias]

Christ, give it a break. I know there's an anti-anti-Microsoft backlash here, but for fuck's sake all he did was mention the previous employer with absolutely NO bias or connotations. If the guy had been employed at XYZ University, I'm sure it would have still shown up.

Not earth shattering, but useful

[f00zbll]

Credit to the guy for thinking of it. It could save a person the hassle of looking up all the address in mapquest. I've never had the need to do such a search on google, since it's easier to just do a yellowpage search. Most yellow page sites like superpages and switchboard already provide that kind of functionality. Google's directory search doesn't have search by distance yet, but I'm guessing it will be added in the future. They kinda have to considering the other directory sites have those features.

Service already exists

[khendron]

There is already a service like this at www.lasoo.com This service lets you enter an address and a business type, and will find all instances of that business within a certain radius of the address.

Last time I used Lasoo was on Mother's Day, to find the closet florist to my mom's house.

Re:Service already exists

[MullerMn]

The difference is that the service you're thinking of probably works from a pre-specified list of locations for the businesses it covers.

The cool thing about the winning google entry is that it actually deduces the location of the search result by finding and parsing any address information that appears on the site!

I think that's pretty clever. - Does anyone know if it's limited to the US?

--
Andy

Re:Geographical Approximation

[Peyna]

Sounds like it wasn't doing IP Addresses or hostnames , but addresses found in text on pages. Using enough rules, and a funky algorithm, you could probably get pretty accurate for a number of pages, enough to produce good results on searches at least.

Re:About the "Former Microsoft Employee" bit..

[Mononoke]

And we certainly dont believe he got better at coding while he worked at MS.

You've never learned from the mistakes of others?

Being a former M$ employee tells me he learned quite a bit.

Nice

[Mr_Silver]

Whilst I'm very impressed with the winner the entry "Robust Hyperlinks" is something that do like a lot.

What would be cool, would be the option to right click on the hyperlink and have the option "Find alternative location".

Or even cooler, have IE (or your favourite browser) on putting up the 404 message have a hyperlink which does the same. Hell, easy enough to do with apache.

Re:Nice

[JFMulder]

There is already something a bit like this with the Google toolbar, it's called "Cached Snapshot of Page". If you hit a 404, just right click on the mouse and chose "Cached Snapshot". If google still has the page indexed, then it has a cached copy of the HTML.

I knew I should have patented it...

[Usquebaugh]

I was thinking about doing exactly the same thing, a common thought?

But the idea of using it just to find business within a certain radius is very limited thinking.

Mobile phones will soon be broadcasting their position. You want interactive guided tours of a city? How about playing full size monopoly? Driving directions? Any sign you currently see could be removed and replaced with a virtual sign? Any number of VR worlds played out in meat space? etc etc

I think that the ability to automatically tell someone where you are will prove to be a boon.

Kudos to the developer for carrying through, rather than my lazy ass postulating :-)

runner up

[gnugnugnu]

the runner up entry

Zhenlei Cai, for his project, Discovery and Grouping of Semantic Concepts from Web Pages with Applications. This effort processed a corpus of documents and found words and phrases that tend to co-occur within the same document, producing a list of pairs of terms that seem to be closely related (such as "federal law" and "supreme court", or "Bay Area" and "San Francisco").

sounds a lot like Google sets

Robust Hyperlinks has to be my favourite.

NetGeo

[*xpenguin*]

There's a public database called NetGeo which will convert IP addresses to latitude and longitude locations. I created a script called IP-Atlas to get a visual location of the lat and lon coords.

Re:NetGeo

[Rayonic]

Nice site, but unrelated to the winner's entry. The IP address will only tell you where the server is, and not the location of, say, a remote business whose site is hosted on that server. In fact, an IP-address-derived location would only get in the way of his project.

Re:About the "Former Microsoft Employee" bit..

[PrimeEnd]

If the guy had been employed at XYZ University, I'm sure it would have still shown up.

Actually he was employed by XYZFind Corp. Literally. And it didn't show up.

Re:About the "Former Microsoft Employee" bit..

[liquidsin]

the part about him being a former MS employee was directly quoted from the submitter, not inserted there by michael. I know it's slashdot reader policy to not read the story, but at least read the fucking summary on the front page before flaming away...

Re:saved a country

[jeti]

It could have saved the US.
US gov has been trying to
capture Bin Laden before 9/11.

At least some changes would
not have occured that fast.

Markov processes

[dukethug]

A Markov process is basically a series of random variables where the value of random variable X^(i+1) only depends on X^i. The idea is that if you want to predict the value of X^(i+1), all of the information you could possibly use is in the value of X^i.

Lots of processes are Markovian- for instance, a random walk. If you're at point x at time t, then you know that there's a fifty-fifty chance you will be at x-1 or x+1 at time t+1. Knowing all of the previous points along the random walk won't help you predict the next point any better than that.

Re:Some might think..

[GuyMannDude]

Though their operating systems may be riddled with bugs and security flaws of all sorts, look at their applications. They tend to be the epitome of quality software.

Yeah, right. That one dancing PaperclipDude was the "epitome of quality software".

Me: (starts writing a letter in Word)

PaperclipDude: "Hi there! It looks like you're writing a letter!"

No shit, Shirlock. What gave it away? The "Dear Sirs" opening line? Shees.

GMD

Daniel Egnor's "Iocaine Powder"

[po8]

In a weird coincidence, I just spent a half-hour last night lecturing about Daniel Egnor's Iocaine Powder , winner of the First International RoShamBo Programming Competition. Credit this guy with two award-winning pieces of extreme programming cleverness!

More Information About the Winner

[td]

I've met Dan Egnor, and this isn't the only cool thing he's done. He's the author of Iocaine powder, the world champion rock-paper-scissors program. He's also the proprieter of sweetcode a web log devoted to innovative open source projects (i.e. projects that don't just clone or tweak existing software.) But his best hack (not described on line, as far as I know) is a version of Pac Man that runs on a PDA and uses a GPS for a user interface -- if you run around an open field carrying the GPS+PDA, the pacman correspondingly runs around the maze chasing Blinky, Stinky and Dinky (or whatever their names are.)

Re:About the "Former Microsoft Employee" bit..

[jfedor]

FYI, Michael Abrash once worked at Microsoft, then went to id Software, and then left id and went back to MS.

So I think there are some programmers at Microsoft that you could learn from (not by seeing their mistakes).

-jfedor

Re:good news for open source?

[rmohr02]

It was at 2 when I replied.

Re:Anyone here who got the CDROM with data mailed?

[polymath69]

Grrr.

I requested the data CDs on February 6th, and got an acknowlegdement email from Google the same day. But I never received the CDs, either, so more or less forgot about the contest until now. And I'm in the US, so they're not discriminating against Germans.

Some fine way to run a contest!

So I'll second the question... did they follow up with anybody?

Why I never would have written that program.

[blair1q]

How many hosts implement their coordinates in their info any more?

5%? 10%?

80% omit it because admins are lazy, and 10% omit it for security reasons.

So Google just gave an award to a tool with half the batting average of a bad baseball player.

--Blair

Re:Why I never would have written that program.

[radish]

RTFA - it's nothing to do with the hosts record. It parses the addresses from the pages themselves.

Re:About the "Former Microsoft Employee" bit..

[asqui]

The reason I included Microsoft Corp. as a former employer and not XYZFind Corp. is becasue I wanted to point out that despite what most of you like to think, intelligent people do work at Microsoft.

Yes really, it's not a large room full of monkeys!

Smooth Move Google

[Uttles]

Did you all read the honorable mentions? Google stands to make some good money off of the ideas and implementations these folks have come up with. I'm assuming that all entries now are owned by Google, and man they might have some really cool new features after seeing the projects that were submitted. I only hope that they give at least some royalties to the developers.

I'm a little disappointed.

[guttentag]

Daniel received a bachelor's degree in Computer Science from Caltech in 1996. He has worked for Microsoft Corporation and XYZFind Corporation, and currently resides in New York City working for a large investment bank.

I was hoping the contest would create new opportunities for some young unknown, like: "Bob is a high school sophomore and currently resides in his parents' barn in Fargo and earns his keep stocking shelves at Toys 'R Us." Oh well, maybe next year.

I wonder what this person worked on at Microsoft..

[zeno_2]

They have a product called Streets and Trips. You can enter in your address, and find out what is within a 5 mile radius lets say. Sounds pretty much like what this guy did.

Speech Recognition

[ZigMonty]

Zhenlei Cai, for his project, Discovery and Grouping of Semantic Concepts from Web Pages with Applications. This effort processed a corpus of documents and found words and phrases that tend to co-occur within the same document, producing a list of pairs of terms that seem to be closely related (such as "federal law" and "supreme court", or "Bay Area" and "San Francisco").

Am I the only one who thinks this would be useful for speech recognition? If you just detected a "federal" and you have two possibilities for the next word, "law" and "paw" say, the software would know it's more likely to be "law". Federal paw is probably fairly uncommon and yet this is exactly the mistake that current software makes.

Wired article

[scubacuda]

Here is a Wired article on it.

Re:winner is...

[redhatbox]

n0 u 533, h3 0wnz0r3d g00g13 w14h h15 31337 m1cr0$0f4 h4x0r1ng 5ki11z!!!

God, help us all... I *knew* we couldn't go an entire story without someone freaking out about the whole "used to work for Microsoft bit."

Ho hum, back to my OBSD boxen...