Google Would Beat Bing At Jeopardy, Says Wolfram

destinyland writes "Stephen Wolfram, the physicist behind the Wolfram Alpha 'answer engine,' believes that Google would beat Bing in any contest based on questions from Jeopardy. 'Wolfram took a sample of Jeopardy clues and fed them into search engines,' explains one technology blog. 'When it came to the first page, Google got 69 percent correct, just beating Ask with 68 percent and Bing on 63 percent. ... To put that into context, the average human contestant gets 60 percent of answers correct, while champion Ken Jennings has a record of 79 percent.' Interestingly, Wikipedia came in last, scoring 23%, though they may have more to do with how Wikipedia handles searches. In two weeks, IBM's Watson computer will compete on Jeopardy against two of the show's all-time human champions."

Comments

Google results still much more accurate

[SuperKendall]

For the past few weeks I've switched over to Bing as my primary search engine.

Overall it works OK, but there have been a number of instances where Google has produced some dramatically better search results, as it in found something related to what I was looking for at all, on the first page. I've only gone over to look at Google when it seemed like the Bing results were not what I was expecting, but it has been interesting to find there still is a pretty large quality gap as I was thinking it might have been closed by now.

Re:Google results still much more accurate

[SuperKendall]

I switched for two reasons:

1) Because I wanted to see if other search engines could work as well

2) Primarily, because I differ too greatly with Google at this point philosophically on the killing of the video tag under the guise to move to an open codec, and I wanted to reduce support of Googles revenue stream, even if only a tiny fraction they will never notice - it just makes me feel better.

Mostly it doesn't matter much, but there are a few times a week at least I have to turn back to Google.

Re:Google results still much more accurate

[MoeDrippins]

> e.g., I search for intel drivers, and I get three pages worth of intelligence tests from stupid quiz sites)

I'm going to have to call shenanigans on at least one point; I just did a search for "intel drivers" (no quotes), and the entire first page was ... Intel Drivers related.

Here's a screenshot: http://img508.imageshack.us/img508/1377/20110130184253.png

Re:Google results still much more accurate

[MightyYar]

I differ too greatly with Google at this point philosophically on the killing of the video tag under the guise to move to an open codec,

I often vote with my wallet, too. I was a NY Giants fan until I witnessed a parking attendant hit a squirrel with his truck. I was appalled by such animal cruelty, and have since switched my allegiance to the Philadelphia Eagles.

Re:Google results still much more accurate

[SadButTrue]

I switched for two reasons:

1) Because I wanted to see if other search engines could work as well

2) Primarily, because I differ too greatly with Google at this point philosophically on the killing of the video tag under the guise to move to an open codec, and I wanted to reduce support of Googles revenue stream, even if only a tiny fraction they will never notice - it just makes me feel better.

Mostly it doesn't matter much, but there are a few times a week at least I have to turn back to Google.

I couldn't agree more. The way google is forcing Apple and Microsoft to not support the open and non patent encumbered WebM makes me sick. It is amazing to me how many sheeple still support evil google over the icon of fair market practices that is Microsoft.

Re:Google results still much more accurate

[GNUALMAFUERTE]

Dude, I hope you are a very clever troll.

microsoft AND apple are trying to kill the video tag with the patent-encumbered h.264. Google is saving it by offering WebM. ALL BROWSERS except for microsoft's explorer and apple's safari support Google's move. You can say whatever you want about Google regarding any other aspect, but in this case, they are doing the right thing.

Re:Google results still much more accurate

[satuon]

If you abandon Google search because of how they handle their browser (or Youtube?), then why aren't you abandoning Bing because of Microsoft Windows, IE, or the doc format?

Re:Google results still much more accurate

[delvsional]

I often vote with my wallet, too. I was a NY Giants fan until I witnessed a parking attendant hit a squirrel with his truck. I was appalled by such animal cruelty, and have since switched my allegiance to the Philadelphia Eagles.

This is slashdot god dammit. I don't give a rats ass about baseball.

Standard Deviation?

[Kensai7]

Aren't these percentages too close to be meaningful? Of course it depends on the sample, but I think unless we get an all-winning AI it's interesting but nothing really special.

That may be, but -

[93 Escort Wagon]

Wikipedia would KICK *SS in the "Anime" category!

Let's do a test.

[Bob Cat - NYMPHS]

http://www.google.com/search?q=stephen+wolfram+is+famous+for+this+self+aggrandizing+book

http://www.bing.com/search?q=stephen+wolfram+is+famous+for+this+self+aggrandizing+book

Google 1, Bing 0

Re:Let's do a test.

[nathan.fulton]

This post is now the top result on Bing. Win.

Aside: Thees type of tests -- where you ask questions specific ways and gauge results -- are really useful if you'd like to do some experimentation with different search engines and avoid "bias." When I first tried Bing, I was astounded at how terrible it was. But my search results improved significantly when I stopped using "Google idioms," phrases that I know from past trial/error are very likely to get me a certain type of result from Google.

Switching search engines for a week is an interesting introspective exercise.

Wikipedia search is useless

[Anonymous Coward]

Unless you've got the exact title, you pretty much need to google site:en.wikipedia.org in order to find what you're after. Google and Wikipedia together work great.

Sure, but...

[realyendor]

...did any of them actually answer in the form of a question?

Yeah, but . . .

[Seumas]

Which would win at Wheel of Fortune?

Re:So what about

[Anonymous Coward]

I can only assume you've never used Wolfram Alpha, if you had you'd know it would score somewhere in the area of 0%

Wolfram

[rossdee]

I thought Wolfram was called Tungsten in America?

Re:Bing queries Wolfram Alpha

[nathan.fulton]

That it's not a search engine.

Missing the point

[kyle5t]

It's not really very interesting whether the facts needed to give an answer are contained in the first page of Google or Bing search results. The difficulty is in understanding what the clue is actually asking, and answering in a way that isolates the relevant information (in the form of a question, of course). And doing so very quickly, even when there is often clever use of language going on. The difference between an "average human" at 60% and Ken Jennings at 79% is huge! And it's not just about how large a base of knowledge you're working from.

Re:Google is a superset of wikipedia

[Missing.Matter]

Agreed, I'm not the best speller in the world, and Wikipedia won't give me anything remotely close to what I'm looking for, even if I'm a letter off. I find myself seaching Wikipedia via Google more often (site:wikipedia.org) because the search results are just plain better.

Duck Duck Go

[gQuigs]

I'm using Duck Duck Go more and more. I wonder how it would fare in this comparison... especially because I find it the best way to search Wikipedia.

It also happens to be great for privacy and a lack of a tracking.

Re:But who better?

[ElectricTurtle]

The problem with correction is that it's not even borderline intelligent. I've been doing something of a personal project creating artist description stubs for obscure demosceners on last.fm, so I've been running into this a lot lately. I'll type in an artist name like 'Cyanid' which Google thinks I must mean 'Cyanide' except that's not the whole search string, which will be like 'Cyanid "person's name"' and Google will search for 'Cyanide "person's name"' and display 0 results. But when I tell it, no, I really did mean 'Cyanid', there will be several results. It really shouldn't be that hard to write the correction code such that it compares the "corrected" search string with the original and at least skips the "corrected" if the return is fucking ZERO.

No current search engine can play Jeopardy

[Anonymous Coward]

Having the correct answer show up in the first document is hardly Watson-like. It still requires a person to plow through the document and find an answer and then determine if it's right or wrong.

Of course none of these search engines nor Wolfram could play Jeopardy. It's one thing to try to come up with the exact specific answer that Jeopardy demands. You also have to have a good sense of when you know the correct answer and when you don't so you know when to try to buzz in. If you buzz in on every question and only know half of them, you will be slaughtered at Jeopardy.

That's what differentiates Watson. It has a very good idea of what it knows and what it doesn't.

Re:Google is a superset of wikipedia

[AmberBlackCat]

What got me was the quote, "Interestingly, Wikipedia came in last, scoring 23%, though they may have more to do with how Wikipedia handles searches". As if the score of Bing and Google doesn't have anything to do with how they handle searches.