Search Engine Learns From User Feedback

An anonymous reader writes "Ian Clarke, founder of the Freenet project, has set up a web search engine that allows users to rate each of the search results it returns. WhittleBit will use your feedback to determine which keywords should be added or removed from your search, then you can search again to get more accurate results. This could be useful for those cases where Google just refuses to return the search results you want. Could improved interactivity be the next big search engine advancement after Pagerank?"

Comments

no it won't replace google.

[garcia]

Could improved interactivity be the next big search engine advancement after Pagerank?"

In short, no.

I have tried Whittebit before (a user had a link to it in his .sig on Slashdot). I was unimpressed with the results the first time (there were 8 or so to work with) and limiting with the thumbs down was of little use when there were so few results.

I can't see google's superiority being challenged by this at all. What else would Whittebit offer me other than this "feature"? I didn't see anything else when I used it (and in fact, was rather annoyed by the fact that it remained at the top of the screen while reading the link I was sent to).

No thanks, just my worthless .02

Re:no it won't replace google.

[Anonymous Coward]

I own the patent to this idea.

Signed
SCO

Re:no it won't replace google.

[nate1138]

Perhaps one reason there were so few result returned is the fact that this seems to be more of a proof of concept than a fully functioning engine. Imagine combining a feedback mechanism with an already excellent search like Google. This can't stand alone, but it would be an excellent addition to an engine that already has a huge index.

One thing that does worry me, what about the potential for abuse. Something like a script that connects to whittlebit, searches by a keyword important to your industry, and gives all of your competitors links thumbs-down.

Re:no it won't replace google.

[Anonymous Coward]

buddy, there's no way it can replace google. it is google. If it wasn't obvious enough that it's mentioned, you can place search terms in google and whittlebit and see for yourself.

I prefer Google, but whittle bit is cool because you can type something in like your name and whittle it down so that you're the only result, if you pick the right ones to cancel. "I am the only [xxx] on Google!"

Re:no it won't replace google.

[enomar]

I agree that Whittebit will most likely fail at overtaking Google, but this feature, or ones like it, could still take hold. What is stopping Google from implementing or buying this technology if it turns out to be useful. I think Google's search results would improve if they allowed for certain user feedback. Perhaps they could keep track of which link each user clicks after searching for 'sex' and incorporate that information into their alorithm. People/bots could try to promote their own sites, but Google could account for this as well. It would also be neat if they could incorporate some of the algorithms used for weeding out spam to filter bad search results.

Re:no it won't replace google.

[bareminimum]

Just download the google toolbar and you'll see that our favorite search engine already tracks results in that way. What they do with the information is another question..

Re:no it won't replace google.

[Baron_Yam]

I think improved interactivity COULD be the next big advance. The issue is how to control the input.

Now, if Google had a subscription system, where subscribing members were randomly given limited moderator and metamoderator privileges...

Re:no it won't replace google.

[10bt]

One thing that does worry me, what about the potential for abuse. Something like a script that connects to whittlebit, searches by a keyword important to your industry, and gives all of your competitors links thumbs-down.

pretty much anything can be abused or hacked, it is a matter of will. google pagerank is being exploted as we speak (or in this case, post).

Re:no it won't replace google.

[nate1138]

Very true. Various link farms (like SearchKing) conspire to affect pagerank. This, I believe, was at the heart of the whole Searchking noise earlier this year. Google reduced the importance of link farm links in the pagerank algorithm, thereby causing Searchking's ratings to plummet.

If the folks at WhittleBit are smart (and Ian Clarke is, very), they will only let the thumbs up/down ratings affect the current session. Or maybe if the use a login or cookie to uniquely identify users, they could save the ratings for the current user only. Then you couldn't affect other users ratings. At least not AS easily ;-).

Re:no it won't replace google.

[Anonymous Coward]

At this point it can't hope to compete with google since the results are being provided by Google. I compared several searches and the sites returned the same top 10 matches for each search. I then selected thumbs down on a couple of words and got a list "whittle" words. Copied those words to my original search in google and placed a - in front of them and whalla same results again. System is nothing more than a Google frontend.

Cool, but can't last

[Dr. Transparent]

Great idea until the second month when your local viagra spammer's SEO guy moves all his pages to the top of the search for "Futurama" or "Ninja Turtles."

Re:Cool, but can't last

[xyvimur]

And that's why it will not suceed. Everything where users are given enough privileges can be turned into `unusable crap' by a group with `bad-intentions'.

Re:Cool, but can't last

[saskwach]

I think this is for whittling down a person's individual searches. My preferences when I'm searching for something about rj45 plugs won't affect yours. This could be cool if used in conjunction with pagerank, so that I don't have to keep clicking on all the little "o"s...it makes it so I only have to see 1 page of links.

The biggest flaw I can see with this system is that if I'm looking for something rare and specific, once I find it, I won't thumbs-up it, I'll just click on the link...It might be useful to have a "thumbs-down all on page checkbox" which might narrow the search intelligently.

Re:Cool, but can't last

[agurkan]

Maybe the search results page can provide a link with a title something like "looks like it", in conrast to "let me see". If the summary provided looks useful you go with the former, if you don't have an opinion, you go with latter.

Re:Cool, but can't last

[hugosantos]

that's why i don't believe this kind of system may work without individual personal certification, something like an advogato-like certification. in theory, a spammer wouldn't be certified by most "well certified" people, making his ranking worthless.

Re:Cool, but can't last

[agurkan]

It is possible to delay the serving of pages that require interactive action. Then an automated robot will not be very fast.
Also, the behavioural pattern of an automated robot can be detected very easily, imagine a connection from a domain suddenly submits favorable reviews for a particular page, and no other such review is submitted. This should raise a red flag. If the effect of reviews is processed and used after an analysis, I think robots can be defeated.

Re:Cool, but can't last

[nerph]

If you combined the pagerank with the personal "whittling" and a statistical engine a la Amazon or Netflix, then it might be an impressive step in search engine evolution.

Then again those statistical engines bring privacy concerns, and we all know how much this crowd loves that! :)

Re:Cool, but can't last

[dracocat]

You are totally off track. The ranking is for your-self only and has no effect on other people. So, the only effect that would have would be for the spammer to be spamming himself.

Re:Cool, but can't last

[Pooh]

why the hell "ninja turtles" could reach top search?

is vanilla-ice just learned perl and own a beowolf cluster?

next things: "4 non blondes" or "goonies" in top 10 at google?

As long as...

[vasqzr]

Ad revenues have nothing to do with the ratings....

All the good search engines end up corrupting themselves (by making money, which I guess is the point of anything...)

Re:As long as...

[bnet41]

Find a way to pay for it, and then ads won't be needed. All those servers and bandwidth cost a lot of money. So do employee's. I really don't think there is a good way to open source a search engine. There are too many physical costs, and I don't like the idea of a Distrubted search engine either.

Nagging

[TripleA]

If they want people to actually use this, they have to come up with some better way of collecting feedback, that scoring bar that remains on top is very irritating.

Kaltix

[bmongar]

I think something like what Kaltix is trying has a better chance of replacing Google. However I don't see that happening either. I just think Google will learn from the user based systems

I doubt this will fly

[The Bungi]

People have been doing experiments like these since the first search engine was rolled off the assembly line. They're prone to abuse and dependent on the goodwill of the user. Imagine of PageRank was based on this - that "SearchKing" dude would have a bot searching for crap and then voting "yes" every time.

Won't work. Goodwill as we knew it in '95 is gone from the Internet.

Re:I doubt this will fly

[Usquebaugh]

Goodwill left the internet with the introduction of http/html.

Re:I doubt this will fly

[Anonymous Coward]

> Goodwill left the internet with the introduction of http/html.

I had no idea they had a gopher or public anonymous FTP document archive... But they still seem to be right here regardless...

Re:I doubt this will fly

[Anonymous Coward]

Jokes don't work so well when one screws up the TLD, sigh. The fix is obvious...

Re:I doubt this will fly

[enomar]

How can you blame lack of goodwill on a protocol or markup language? Lack of goodwill ended when people found out they could make money using the Internet. Greed, not technology, killed the goodwill.

Re:I doubt this will fly

[Henk Poley]

You apparently never used SubMe? It worked quite good. It really helped me preparing for school works.

To bad they took it down. I've tried to email the authors (SmartHaven.com), but they use a blacklist email filter that blocks my ISP for some obscure reason :-(

Re:I doubt this will fly

[cK-Gunslinger]

perhaps he was just refering to the timeframe when http showed up, not the technology itself.

Re:I doubt this will fly

[Usquebaugh]

Which technology was the trigger that allowed greed to become profitable? It is also the point at which Joe Public showed up on the net. The dream died there and then. A horrible but accurate demonstration that mankind has not evolved in millenia.

Re:I BELIEVE IN GOD

[Washizu]

"Just like birds can never understand how to make a pizza"

The pizza I ate last night tasted like it was made by birds.

hell no

[Anonymous Coward]

no, i dont want to have to give feedback in a search, I just want to type keywords and find related results ...

Re:hell no

[eggarsuit]

I agree. The #1 barrier to the success of this kind of technology is user laziness. When you visit a search engine, you don't want to be the one to provide results.

I like it.

[Doesn't_Comment_Code]

I like the idea of interactive page rankings. I don't think it should be the one decisive ranking alogrithm. But human interaction is just what search engines need.

I do a lot with Google, and it leaves some to be desired. The goal of Google is to make the ranking of pages partly out of the hands of webmasters, so they can't just trick the spiders. And that has worked very well for Google (serves over 70% of internet searches). But all page ranks are very cold and calculated. Maybe that cold, calculated rank is a good place to start, and then it's time for human reviewers to fine tune the list.

By the way, Google has attempted to acheive this concept of human ranking by watching to see how long you stay at a page you clicked on. If they rank a page 1, and you click it, and immediately return to the search page, they penalize that page. So if even Google is trying the same abstract concept, it probably has a future on the web.

Re:I like it.

[Thoguth]

By the way, Google has attempted to acheive this concept of human ranking by watching to see how long you stay at a page you clicked on. If they rank a page 1, and you click it, and immediately return to the search page, they penalize that page. So if even Google is trying the same abstract concept, it probably has a future on the web.

If that's true, then the way I do searches is counter-productive. I load the google search page, and then middle-click all the links that look the most promising and read them in tabs. No wonder Google's searches have seemed to get worse and worse for me lately, I'm training it to think my most promising results are no good!

Re:I like it.

[Anonymous Coward]

>I like the idea of interactive page rankings

So do I!

My site is pretty cool *click*
My site is pretty cool *click*
My site is pretty cool *click*
My site is pretty cool *click*
My site is pretty cool *click*
My site is pretty cool *click*
My site is pretty cool *click*

Wahey - i'm top of the list! Kewl!

Re:I like it.

[Anonymous Coward]

But it sounds like Google is already doing it better than WhittleBit. The system that you describe is much harder to abuse since it is analyzing what the user actually does rather than what the user says. The only case where I can imagine the Google technique failing is if the user clicks on a site that s/he likes, but has already seen, then immediately clicking back to Google. Also, doesn't Google already incorporate sites from dmoz.org? That is another 'human touch.'

Re:I like it.

[JimDabell]

By the way, Google has attempted to acheive this concept of human ranking by watching to see how long you stay at a page you clicked on. If they rank a page 1, and you click it, and immediately return to the search page, they penalize that page.

Do you have a reference for that? According to the HTTP RFC, user-agents aren't supposed to talk to the server when users hit their back buttons, but rather display what the user last saw (despite it possibly being stale). It seems odd that Google would try to circumvent this, especially as there isn't a reliable way of doing so.

Re:I like it.

[costas]

Well, if you're excited about user-rankings and feedback, check out the newsbot in my .sig. It focuses on user interaction with the code/algorithm to build not just page rankings but also relationships --between pages, and between users. Try it out, I am guessing you'll like it...

Re:I like it.

[Anonymous Coward]

Unless Google is giving you results that are actually redirects, then you aren't telling them anything when you click on those URLs.

I know Google has done this in the past, but I haven't seen it happen in a long time. I tend to rewrite obvious redirector URLs anyway. They can track someone else, thanks.

Re:I like it.

[Anonymous Coward]

Let me introduce you to the concept of "cookies." They store information about you in a flat file on your computer. This way when you return to their site they can tell how long it has been since you were last there and what you clicked on when you left.

Re:I like it.

[blibbleblobble]

"I like the idea of interactive page rankings. I don't think it should be the one decisive ranking alogrithm. But human interaction is just what search engines need."

Big question is: how does it avoid Google's problems, (i.e. how does it perform when millions of people are spending big money trying to distort and to cripple the results?)

If feedback were ever tried, it would certainly need to be personalised, and the sharing of page-rank limited to "people who voted similarly to you in the past thought *these* results would be useful". That's a hard enough problem that even Google can't do it with thousands of clustered PCs; the computing power for personalised searches is quite considerable.

But allowing people to naively rank searches just leads to thousands of low-paid people working day and night to rank their employers' sites the highest. Or worse, if there was no Turing test on the inputs, millions of scripts competing in a bizarre worldwide version of CoreWars to take control of the searches.

Personalised searches? Will there ever be enough feedback for it to work on any but the most common search terms?

Re:I like it.

[Thing 1]

I load the google search page, and then middle-click all the links that look the most promising and read them in tabs.

I do the same, but I don't think Google "knows" that you're doing that.

As far as I know, Mozilla doesn't send Google any info when you middle-click (or even left-click) on an outside link (if you clicked on the cached link, then of course it does).

So if you never refresh the Google tab, it doesn't report anything of your Googling habits back to Google. So you're not training it at all -- as opposed to people who follow the link in the same tab and come straight back, or don't come back.

Slightly off-topic, I hate when a page refreshes when I hit "back." It should always get that page out of my cache, and only refresh it if I click "refresh" (or Crtl+R or F5). It's a waste of bandwidth and it slows me down. </rant>

Re:I like it.

[neurosis101]

How about the real problem: it only takes a few morons to mess with the ranking system to really mess it up. Can't you see a group of people adding "penis" as a keyword to everything?

Re:I like it.

[Christianfreak]

Probably not. I know that Google sets a cookie, most likely they store what you clicked on in that cookie or in a session ID associated with it. It would read those cookies regardless of weither or not it was in a new tab/window/whatever.

What is probably more counter-productive would be to turn off cookies, but even then without the information, Google wouldn't get worse, I would think it would just return the same results everytime.

Re:I like it.

[plonista]

This alone will not cause a problem, but if you (as I often do) immediately click open the next search page in a background window, you'll tell Google that you were not interested in those other links that you have not even looked at yet.

I've been getting the feeling that google's results have not been as focused and well ordered for me as they used to be. I'm going to clear cookies and see if I see a difference.

Re:I like it.

[JimDabell]

Let me introduce you to the concept of "cookies."

Before acting so condescending, please be familiar with the subject matter and think about what you are saying. It seems you are doing neither.

This way when you return to their site

My point is that clicking your back button shouldn't return you to their site in the way you are thinking of. It should show a stale copy - therefore no talking to the server, and no transmitting of cookies.

Re:I like it.

[Hatta]

But doesn't the cookie scare you? I find it terrifying to think that google has a database of every ip and all the searches performed from that IP, and that's collected even without cookies that never expire. Remember that google is a private corporation, and is likely to cow to any request for information by a law enforcement officer. I'm sure it won't be too long til all of this is linked straight into TIA. Check out Google Watch for more info.

Re:I like it.

[Anonymous Coward]

Dude. Where does he mention the back button anywhere in his post? You keep assuming that something is happening when the user presses the back button and I am telling you it is not necessary. And if they did hit the back button and got a static page with no communication to the server they would have to click another link at some point, wouldn't they?

You are also forgetting the whole google bar thing. I don't mean to sound condescending but maybe if you thought out of the box for a second you wouldn't feel the need to doubt the original poster.

Re:I like it.

[JimDabell]

Where does he mention the back button anywhere in his post?

What do you think he means when he says:

If they rank a page 1, and you click it, and immediately return to the search page, they penalize that page.

How would you immediately return to the last page you were on?

And if they did hit the back button and got a static page with no communication to the server they would have to click another link at some point, wouldn't they?

But if Google monitored clickthroughs for this purpose, then people like me who open half a dozen links from the results page in new windows/tabs all at once would screw up their calculations.

You are also forgetting the whole google bar thing.

Well no, as I specifically said that there was no reliable way of doing it. For instance, since the Googlebar only works with Internet Explorer, if they used it to track users in this way, it would bias rankings against sites that don't work well in Internet Explorer, but not bias rankings against sites that don't work well in other browsers (since only the first class would give negative feedback). Whether such biases are acceptable or not is obviously a matter for Google to decide, but as I said, it's a bit odd.

maybe if you thought out of the box for a second you wouldn't feel the need to doubt the original poster.

I thought it was odd, and asked for a reference. I then defended myself when somebody who missed my point entirely tried to patronise me. That is all. Nothing to do with "thinking outside of the box", I just wanted more information on it.

Re:I like it.

[Doesn't_Comment_Code]

That is pretty cool... especially for news. I always hunt and pick through the newsfeeds for which stories I want to read, and it would be really nice to have a search engine help you out.

Re:I like it.

[Doesn't_Comment_Code]

I've never heard or seen exactly how they do it. I've wondered that before as well. I always assumed that they checked to see if you clicked another page on their search results page soon after clicking another. But even that wouldn't work unless they used some type of redirect to track the click(and we know they don't use javascript).

Re:I like it.

[sircrown]

By the way, Google has attempted to acheive this concept of human ranking by watching to see how long you stay at a page you clicked on. If they rank a page 1, and you click it, and immediately return to the search page, they penalize that page. So if even Google is trying the same abstract concept, it probably has a future on the web.

They've gone one step further and implemented the exact same thing whittlebit is providing and it's been around a lot longer too. If you use IE, just install the Google Toolbar and enable the Voting buttons. From the toolbar help:

Voting Buttons - You can vote for or against a page or search result by using these buttons. Click the happy or unhappy faces to tell Google that you like or dislike a particular page. These buttons also can be used to report especially good or bad results after searching with Google. Indicate satisfaction or dissatisfaction with your results by clicking the appropriate button while still on the search results page. This feature is in test mode for now, so you won't see any immediate effects by voting for or against a page.

Re:I like it.

[Anonymous Coward]

He said "immediately return to the search page"

You said "immediately return to the last page you were on"

Do you see the difference there? And I am not saying that it is an exact science. Obviously it is not going to be reliable - search engines by their nature are not reliable. That is the whole point.

And judging by zeitgeist IE still accounts for over 90% of their hits. I don't think anybody needs to worry about there being a skew from non-IE browser users.

I am not saying that this does exist in Google or that it is a good idea or not. I am telling you that you misread (and seemingly continue to misread) his original post. Would you like me to create a proof of concept for you?

Re:I like it.

[JimDabell]

He said "immediately return to the search page"

You said "immediately return to the last page you were on"

Do you see the difference there?

Look at it in context. There is no difference:

If they rank a page 1, and you click it, and immediately return to the search page

Step through it.

1. Search for something
2. Get results page
3. Click a link
4. Immediately return to the results page

If you immediately return from the page that google sent you to, then yes, it's going to be the last page you were on (assuming no author stupidity like redirects that break the back button).

And judging by zeitgeist IE still accounts for over 90% of their hits. I don't think anybody needs to worry about there being a skew from non-IE browser users.

Good point, any end-user feedback is going to skew in this way.

Re:I like it.

[Anonymous Coward]

Use an anonymizer you conspiracy theorist.

Re:I like it.

[mahdi13]

Unless your using Google for kiddie pr0n, system hacks and cracker scripts I think you'll be ok with a little cookie
...as long as it's a sugar cookie, yummy!

But if that is what you are using Google for...Burn in Hell you Communist Nazi Pig!

Re:I like it.

[JohnwheeleR]

this is assuming people are good. what happens when someone writes a script to corrupt the rankings?

Re:I like it.

[Anonymous Coward]

Google Watch is operated by a wanker who was buggered by the rank he was getting. He has An Agenda, with capital A's and all. Keep that in mind.

Re:I like it.

[Anonymous Coward]

By the way, Google has attempted to acheive this concept of human ranking by watching to see how long you stay at a page you clicked on. If they rank a page 1, and you click it, and immediately return to the search page, they penalize that page. So if even Google is trying the same abstract concept, it probably has a future on the web.

I dont think this is true for various technical reasons. Firstly, when you click on a page URL on the google result page, google has no way of knowing which link you clicked as you do not connect to the URL server and NOT to the google server, so it can not possibly penalize that page. In addition, most of the time, when you hit the back button to go back to the search result, it comes from the cache and again not from Google, so Google has no way of knowing how much you spend on a particular page.

I have recently heard a lot of *urban legends* about google (some similar to this one) and most of them are very funny and not at all technically sound. I guess it comes with the territory when something performs as google does :).

Google for obvious reasons do not show all the tricks up their sleeves but after working a littl e in this area my intelligent guess is that besides pagerank, they involve most what can be called human engineering e.g giving links from yahoo.com more weight than from my home page and so on.

What WhittleBit has done is really not something new, any person working in the area of search engine will tell you that these kind of systems have existed even before pagerank. Though I think this may be the first to be actually deployed on the web. Most of such similar systems have been limited to academic circles and researh papers till now.

Re:I like it.

[Anonymous Coward]

It still happens randomly for different users. It was my turn 2 months ago, and all my results were linked through Google for a few queries. I've saved the page as a souvenir :-)

Re:I like it.

[pjwhite]

Kartoo has an interactive ranking system, too. It's an interesting system, drawing a visual map of search results.

Re:I like it.

[Threni]

> But all page ranks are very cold and calculated.

Yeah, I prefer warm, fuzzy human error.

Re:I like it.

[wheany]

Slightly off-topic, I hate when a page refreshes when I hit "back."

Try Opera.

Re:I like it.

[Anonymous Coward]

The only problem with that is javascript popups.

If Google penalizes a page for not being read when clicked on, then the page that shoots up 9999 popups that have to be clicked away and possibly crashes your whole browser from slimy attempts to take it over will not be penalized since it will be impossible for the user to return to the google search page.

Ack! Do you know what you're doing?

[numbski]

This is a great idea in concept, but the potential for abuse is incredibly high (if it's implement on a system that actually matters, like google).

Imagine for a moment, a geek for hire, such as myself, writing a PERL script and deploying it on several servers nationwide. It uses LWP::UserAgent and spoofs a few different versions on IE on Windows. It then run searches for hot keywords that my client wants to rank high on. Then it 'mods down' anything it isn't my client's product, and 'mods up' what is, or links to, my clients products.

Set the script to run several times a day at each location. Write some spyware that does so in the background of a shareware-app-for-hire (Kazaa?).

You see where I'm going with this? Protections would have to be in place.

tweaking for higher results

[Ugodown]

Even though google uses PageRank, often sites are higher in the results are only there because they had the right keywords in the title. Sites like this have been tweaked with other similar tricks to score higher. Obviously, this new system would be able to get around this. Perhaps, when joined with Google, this could take over when PageRank fails to be applicable. Then we would have something great!

Body before it gets /.ed

[nother_nix_hacker]

It was going well until we realised that all people wanted was pron so we just provide that now.

Re:Body before it gets /.ed

[Lu Xun]

Probably much the same could be said about Freenet...

Re:Body before it gets /.ed

[Anonymous Coward]

Let's just hope that it is faster than Freenet. I am still downloading a 1.2 MB file from 3 weeks ago.

Re:I BELIEVE IN GOD

[Anonymous Coward]

I've had BBQ chicken pizza before. Chicken is great on pizza. So birds can actually make very fine pizza.

Yay!

[Anonymous Coward]

Now I can look up porn using synonyms! :D

Re:I BELIEVE IN GOD

[Anonymous Coward]

i've had enough of your lack of respect for birds.
they'll learn how to make pizza eventually, its just taking them a little longer. I mean, monkeys learned how to make it....oh wait, thats after they evolved into us...

too bad for Whittlebit

[Anonymous Coward]

... that they apparently didn't do a patent search before implementing this. I have a patent covering pretty much exactly what this entails (really!)

Similar concept...

[X86Daddy]

I think I found the link somewhere on Slashdot once:

Gnod.net is a learning system like a search engine that allows you to put in your three favorite authors/musicians/movies and it returns a series of "suggestions" that match, asking you if you like/dislike/haven't heard of each result in series.

This sort of creature has the potential of placing the final nails in the media cartels' coffins, as it provides what's missing from current P2P and self-production techniques: a recommendation/promotion mechanism.

Re:Similar concept...

[Henk Poley]

Last.fm (now being bundeled with audioscrobbler.com) has a better system for music rating than GNoD.net. Because it watches what people actually listen too instead of what they think they listen.

I like this kind of "smart systems" or "AI", we just ought to find a better description than those.

No.

[Anonymous Coward]

Does this "search engine" search images? No. Google does.

Does this "search engine" search 20 years of Usenet? No. Google does.

Does this "search engine" provide stock quotes, maps, phone numbers, and news? No. Google does.

Thanks for playing. Google will never lose.

Re:No.

[Anonymous Coward]

I agree. Once a single company has done a task well, there is no need for any other companies to try.

Google has solved the problem of web searching. Please solve other problems now.

Re:No.

[Anonymous Coward]

Any research about web searching must come from Google Labs. Any other research is a threat to Google's dominance and should be treated as such.

Re:No.

[saskwach]

This search engine uses google...look at a search string, it's just a front end.

Re:No.

[utmecheng]

You're an idiot. Those cheap-trick addons only made google better after they had a solid search engine that gained popularity. Any other search engine can add on parts like google does. I dont think whittlebit is really trying to compete with google. futhermore in response to the hundreds of 'hell no i dont want to give input' posts: what do you think clicking on a link from a google search does?? Part of pagerank includes how you respond to the search. if there are thousands of people all going in a certain direction google takes note! It is google's search engine that is so good that will keep it ahead, if they lose that they will not be the diety that they are today.

Re:No.

[Anonymous Coward]

Who needs a "front end" to Google? In my browser (Mozilla, Galeon, Firebird, Opera, Safari, Camino, and others) I have a search box to search Google. Why would I waste my time on this worthless front end that is trying to undermine the Google?

Please boycott this "learning" search engine, and learn to use Google.

Re:No.

[Anonymous Coward]

Please moderate this post down. When you search for anything on Google, you are presented with a list of direct links to search results. You don't get some thinly disguised exit link like http://google.com/redir.asp?page=www.foo.com. No. Instead you get a real link to a real web page. Google can't track whether you have clicked this link or not.

Google has never, would never, and is never tracked where you click. That's important to a privacy-conscious man like myself.

Re:No.

[utmecheng]

thats a good point. i have to admit i was repeating second hand information. but certainly any cached page is logged.

Re:No.

[mhesseltine]

Is this testing a concept? YES

Could something like this be implemented in Google? YES

Is this supposed to replace Google? NO

Are you a troll? YES

Thanks for playing.

Re:No.

[Sanity]

Does this "search engine" claim to reproduce all of Googles features? No

Does this "search engine" allow users to give feedback on search results? Yes. Google doesn't.

Thanks for playing. Google will never lose.

Remember when people said that about Yahoo, or Netscape, or any of the other market dominating companies that were eventually replaced?

Re:No.

[Anonymous Coward]

Thanks for whining, loser. Maybe some day you'll be able to write highly-rated comments by pandering to the lowest common denominator of Slashdot-reading sheep.

In the meantime, enjoy your "offtopic" moderations.

Re:No.

[Anonymous Coward]

Read. Google let users refine its search results more than 5 months ago.

And I don't remember Yahoo or Netscrape (suck suck scuk) having a search engine worth using. Google is better, and it always will be.

Re:No.

[saskwach]

Now that's kind of silly. You won't use it, I won't use it. That doesn't mean that the majority of users, who can't form a good search with two hands and a dictionary won't use it. This appears to be a tool for simplifying the web search process. "Learn to use Google" isn't valid because many users couldn't care less about learning what +,-, and "" mean.

Re:No.

[jcast]

Odd. He seems to have gotten a +5 rating.

Re:No.

[Anonymous Coward]

That's okay. "Funny" moderations don't affect karma. You have to be wise, not a wise-ass.

Thumbs down....

[pjdepasq]

Warning: fsockopen(): unable to connect to 127.0.0.1:9182 (Connection refused) in /home/ian/whittlebit.com/wqserver.php on line 13
Connection to WQServer failed

I rate it thumbs down (for now)...

Re:Thumbs down....

[Anonymous Coward]

I think we destroyed it. :-O

One word - "abuse"

[MrFenty]

This will quickly be abused, much like other rating systems like Amazon's book reviews. Anything worthwhile will ultimately be abused, you can be sure of that.

Google is Highly Accurate

[(eternal_software)]

"This could be useful for those cases where Google just refuses to return the search results you want."

That has really never happened to me. Google is fast and extremely accurate, especially when you do a more advanced search, + this and - that.

I'm not sure I would want to take the time to "rate" search engine results and re-search when I can just fine-tune my search from the start.

Re:Google is Highly Accurate

[Henk Poley]

Then you never searched for anything difficult to find. I also know these tricks, but for example, if you want to search for a search engine similar to WhittleBit, what query would you give? No, related:whittlebit.com is not going to give you good/all results.

Hint, try to use "smart search engine" or "smart search tool", find that it includes about any 'tad bit smarter than dumb' search engine.

Have fun :-)

PS: Ever had a searchengine that won't lead you to the homepage of the manufacterer Quantum every time you search something about "quantum computing"? This one will, I hope, just like SubMe did...

Re:Google is Highly Accurate

[sircrown]

PS: Ever had a searchengine that won't lead you to the homepage of the manufacterer Quantum every time you search something about "quantum computing"? This one will, I hope, just like SubMe did...

You got greedy when you threw that last bit in there. I just googled "quantum computing" and quantum.com isn't even in the first 250 results, if it's there at all.

Unless I'm misunderstanding you and you actually want to see quantum.com in which case you're just setting yourself up to fail with that query.

Re:Google is Highly Accurate

[Henk Poley]

Hehe, just thought someone would try to do that :-)

When I used a similar search engine, called SubMe, years ago (2000-2001) I researced a fair bit about quantum computing for a school project. At that time there also was a Dell-alike computer distributor called "Quantum Computers". Leading to bazilion results, which setteled down after 2/3 batches of searching and aproval/disaproval.

Re:I BELIEVE IN GOD

[Anonymous Coward]

Hey assclown maybe you want to tell me where the fuck did all the materia came from???

Ouch - major slashdot - mirror of page

[Sanity]

The server is down - it was totally ill-equipped to handle a slashdotting unfortunately, I was hoping it would get some testing, but this is a bit much ;-)

As a poor substitute to being able to play with it (try bookmarking whittlebit.com and coming back in a day or two) I will try to answer people's questions. For the moment - here is the blurb from the front page:

What is WhittleBit?

Have you ever searched for something and wished you could tell the search engine that it was totally on the wrong track and it should try again? Well now you can! WhittleBit works much like most other search engines, except it can help you to refine your searches by allowing you to give positive or negative feedback on each search result.

Simply rate the search results by clicking on the "thumbs up" or "thumbs down" buttons then click on Whittle to get a refined set of search results based on your feedback.

Tips

• Even if you visit another site and then return, WhittleBit will remember your search query until you explicitly click the "New Search" button.
• You can either rate a search result on the results page itself, or visit the page and rate it using the buttons at the top of the page. You will return to the WhittleBit search results after clicking one of the buttons.
• WhittleBit requires a browser which supports "Cookies" and "Frames" such as Mozilla or Internet Explorer.

- Ian Clarke, creator of WhittleBit

Re:Ouch - major slashdot - mirror of page

[Hatta]

Have you ever searched for something and wished you could tell the search engine that it was totally on the wrong track and it should try again?

Isn't that what the back button is for?

Re:Ouch - major slashdot - mirror of page

[Omerna]

Just a question, no offense intended, but how is this better than than other search engines "Search within these results" button? Narrows the field much like this whittling idea and seems easier, IMO.

Not my damn job

[loserbert]

I want THEM to tell ME what the good results are, not the other way around. If I wanted to do that I'd write my own search engine. Don't bring some lame ass solution where I have to do all the work.

Sounds Great...but

[mstieg]

who wants to wade through results and rank them? I came here to search!

That's why google is king. It doesn't require you to do *anything*. It barely *allows* you to do anything.

And it still returns what you need.

That's the perfect UI.

Re:Sounds Great...but

[jrkotrla]

hmm...

You're aren't required to do anything.... are barely allowed to do anything..... and this is perfect?

you must be a Mac user, right?

Re:Sounds Great...but

[mstieg]

Um. No.

The beauty is: It Just Works.

Searches aren't better when they have more knobs and switches. I've done lots of usability tests on them. They are ignored by the vast majority of people.

As they should be. Make it work and they will come.

Remember how freaked out you were the first time you saw google? But it worked and you kept using it.

Re:Sounds Great...but

[Henk Poley]

Not trying to troll or whatever, but...

Congratulations, you have proven yourself and the rest of the world that you don't need it. If Google is good enough (for you) then why moan?

The guy made it 'all for himself', because he does (like me) find it difficult to find to right keywords from time to time. But because he isn't selfish he wantsto share it with the rest of the us.

Understood?

Re:Sounds Great...but

[hellfire]

you must be a Mac user, right?

I know I am a Mac user.

[sarcasm]
I'm also one of 150 Million Americans actually busy with my life. I don't have time to rank every entry I get on a search. I don't live with my parents and sit at home doing nothing but tweaking my linux box, surfing tech websites, and dissing other people for their choice of technology. I have a job and I expect technology to, well... I dunno... just work?
[/sarcasm]

I have a million gross overgeneralizations, too. I don't believe any of them, including the one I just posted above (and yours), and I don't think any of them are particularly funny. The thread starter had a great point. People just want things to work. I'm sorry if /.ers think that everyone has to be a computer programmer just to use a computer. By that logic everyone should be a lawyer/doctor/politician/gourmet chef/day care specialist as well.

heh - server "whittled"

[cavemanf16]

WhittleBit server whittled down to nothing in 150.030 seconds.

"Free Search" has no place in the commercial web.

[Boss, Pointy Haired]

Google's PageRank is failing miserably for commercial search. PageRank is fine for academic / informational searches.

In a commercial environment, it is simply not possible for a free search service to exist that is fair, represents an even distribution of wealth, and is immune from abuse.

Advertising has to be paid for. "Free Search" is fine for university sites and purely non-profit informational pages, but for a commercial search your position in search engines must be purchased based on the keywords against which you wish to bid.

Otherwise basic economics breaks down.

Re:I BELIEVE IN GOD

[afex]

wait, i though materia was the stuff you put in your sword in Final Fantasy 7

Re:Ack! Do you know what you're doing?

[wfberg]

This is a great idea in concept, but the potential for abuse is incredibly high (if it's implement on a system that actually matters, like google).

Check out the voting buttons on the google toolbar.

great news!

[PhysicsExpert]

This seems like a great idea. Google might be number 1 in the search engine rankings at the moment but it would be good to see them have a bit of competition so that they do not use their dominant position for financial gain.

Here in the lab we're doing some work on using the principles of thermodynamics in order to improve search engines. The second law of thermodynamics states that in a closed system ethalpy will alway increase, which is a lot like the disorder cause by sites spamming themselves to search engines . In addition the searching patterns of users can be thought as analogous to the fermi level of a solid. In theory applying thermodynamic equations to the process of search engines should allow for more efficient algorithms to be developed. Although this has been known for some time the process involves solving some fairly hefty quadratic equations which have needed some serious computing power to process. Hopefully though a real leap forward should be no more than a few months away.

Re:great news!

[hkmwbz]

"Google might be number 1 in the search engine rankings at the moment but it would be good to see them have a bit of competition so that they do not use their dominant position for financial gain."

Sorry to burst your bubble, but Google is a commercial company. Its aim is to make money. I see nothing wrong with Google using its dominant position to make money. After all, it does this because a lot of people think it's the best search engine out there, not because it forces others out of business with shady business practices. It delivers a solid "product" which makes money because it is good, not because it forces people to use it.

One thing is being wary of signs of abuse. Another thing is to think that Google shouldn't be making money just because they are big and dominate the search engine market.

Re:I BELIEVE IN GOD

[Anonymous Coward]

You mean came IN, of course...

thumbs down

[stonebeat.org]

i would have given a thumbs down to this message, but unfortunately there was no thumbs down at all. ;) Warning: fsockopen(): unable to connect to 127.0.0.1:9182 (Connection refused) in /home/ian/whittlebit.com/wqserver.php on line 13 Connection to WQServer failed

how long until

[the_2nd_coming]

how long until google buys them out?

I give it 3 weeks after they begin getting rave reviews.

What is really needed is...

[Anonymous Coward]

What is really needed is to separate out commercial sites. Google works great 90% of the time but when you are searching for something that triggers a response from sites trying to sell something, the results get swamped with the commercial noise.

This would benefit commercial sites because when you really are looking to buy something, you will be guaranteed not to be annoyed by anything non-commercial.

-- YAAC (Yet Another Anonymous Coward)

How ironic?

[CompWerks]

Is it that a google search for whittlebit doesn't even have a link to whittlebit.com.

Pleasure and Pain

[handy_vandal]

What we need are computers that experience pleasure and pain, along with the means to deliver these sensations.

When a search engine delivers good results, the user rewards the engine with a dose of pleasure.

In return for bad results, the user unleashes a blast of pain.

That should teach the circuits a thing or two about delivering the goods!

Re:Pleasure and Pain

[ckessel]

Yea...until you get a system that's into S&M.

Re:Pleasure and Pain

[commodoresloat]

Great idea! Just think of the applications of such technology to pr0n!!!

Kartoo

[SillySlashdotName]

I have used kartoo and like it.

It does not "learn" per se, but allows you to select from multiple possibilities using a GUI - and it has been available for a while.

If I have problems finding something with Google, I use Kartoo.

Re:I BELIEVE IN GOD

[Anonymous Coward]

Let this guy tell you about it.

Fortunately, most God-believers are getting this kind of facts pretty straight, but you obviously don't. Are you one of those guys thinking that the Appolo landings were just hoaxes ?

Something like that

[siskbc]

The biggest flaw I can see with this system is that if I'm looking for something rare and specific, once I find it, I won't thumbs-up it, I'll just click on the link...It might be useful to have a "thumbs-down all on page checkbox" which might narrow the search intelligently.

That would help, but it would have to know why they're bad to know how it would differ from other results that might be more acceptable.

Here's what I would do. First, instead of google returning the most relevant choices, it needs to be a factor of relevance and diversity. So, with the typical "apple" search, it would return some apple computer results, some fiona apple results, and some results about the fruit. All of those would be highly relevant, but it would only give, say, a few of each. You could then click on the more relevant results (if you wanted apple the fruit, you'd click on the three fruit links), at which point it would reject the others and give you more of what you want.

The key here is that it would have to give diversity in the beginning for you to be *able* to differentiate things like what you want from things you don't. This is not how google works now, I don't believe.

For what it's worth, this algorithm wouldn't be too complicated to do. I lack the programming ability, but I could do the algorithm in pseudocode (at point most decent programmers could reduce it to C++). It should be quite possible.

Re:Something like that

[2short]

"For what it's worth, this algorithm wouldn't be too complicated to do. I lack the programming ability, but I could do the algorithm in pseudocode (at point most decent programmers could reduce it to C++). It should be quite possible"

No offense but:
In general, statements like that are used by people who haven't actually thought through the algorithm in detail, or who don't have good knowledge of algorithmic theory.

In specific, your suggestion sounds excellent. Sufficiently excellent that I would be very surprised if Google, with their famously large R&D division, didn't have some very smart people thinking about it or something similar. In any case I'm sure they've got people thinking about some way to deal with what I'll name "The Apple Problem". It's really the main problem with their engine as far as I can tell. The fact that they haven't come out with something encourages my suspicion that this is one of theose seemingly-easy-but-actually-incredibly-difficult deals. Thinking about it briefly the first couple aproaches I come up with wind up being factorial time. Plus there is a lot of fuziness as far as how to promote Fiona Apple links but not just lousy Apple Computer ones, not to mention search terms where the "families" of hits are less distinct than for Apple.

Re:Something like that

[Elwood P Dowd]

Yes, it would likely be very difficult, but Google might have done some of the hard work already. Remember their Google Labs tool that, given a few sample items, would return a list of other things in that category?

If you gave it:
apple
pear
orange

It would return:
grape
cherry
strawberry
kiwi
etc.

If you gave it:
apple
dell
compaq

It would return
gateway
hp
hewlett-packard
ibm
etc

So, if Google's tool could also be used to identify the different meanings that each word has, then maybe they could give you a few links for each meaning. Of course, many search items might not be this easy to categorize. We haven't yet proven that your algorithm is easy, and we also haven't proven that it would be useful for many types of searches.

Re:Something like that

[p3d0]

Hey, that's a pretty good idea.

Re:Something like that

[alekd]

Northern Light used to do this. It worked rather well, but in most cases writing better search criteria is less work. The reason why Google doesn't offer anything similar might be because Northern Light has patented their subject classification and taxonomy.

Re:Something like that

[phazei]

well... then why don't you type "apple +computer" or "fiona apple" or "apple +fruit" ?

If you think of multiple relevant key words you'll usually find exactally what you want within the first page... well, at least that's what I've always done and always gotten. Search engines have really usefull rules, not many but they do help alot, quotes for phrases, or + or - prefixes...

Just a thought

Re:Something like that

[siskbc]

If you think of multiple relevant key words you'll usually find exactally what you want within the first page... well, at least that's what I've always done and always gotten. Search engines have really usefull rules, not many but they do help alot, quotes for phrases, or + or - prefixes...

Right, and I consider myself rather skilled in the art of search engine massaging. :) This more for the folks who aren't so inclined. Even for the rest of us, there are sometimes subtle relationships between results that, even for we "experts," such a system would become a very powerful tool.

Re:I BELIEVE IN GOD

[Anonymous Coward]

You mis-spelled 'God' there tough guy. It is a proper noun and thus, capitalized. LEARN TO SHOW SOME RESPECT you illiterate OAF!

Re:I BELIEVE IN GOD

[Lord_Dweomer]

"The pizza I ate last night tasted like it was made by birds."

Better than my night....my pizza was made WITH birds.

Post-Google Searching

[omnipotens]

I've wished that Google would do this for ages. The possibilites for increasing accuracy are endless with a model for this. I wonder where else this could go. Maybe some sort of integration with another (though possibly encumbered by its relationship with LookSmart) post-google search engine like Grub? However, this is a BIG step. Once information like this begins to be integrated into a massive database, we could see the next quantum leap in search engine accuracy, and possibly breadth. One thing that could help all of this is to watch what it going on by a list-- here is mine, so far:

• Teoma
  Is supposedly more accurate than google, but I've found it to be only okay at best
• Turbo10
  "Searches the deep net" by connecting to site databases to get the most relevant info. A lot of this info, however, comes from Google itself.
• Grub (a project, not an active engine)
  A distributed search engine project. It would use tons of people's computers as crawlers like seti@home
• WhittleBit
  Read the story

So, maybe we'll get somewhere after google (not that google isn't a Good Thing), after all? And.... well, Ian Clarke and his projects is/are/may soon be really rocking the world. Those include:

• Locutus: www.locut.us
  A giant search system for pre-existing content, aimed at corporations.
• Freenet: www.freenetproject.org
  An anonomyous content-storage system that works as a giant encrypted webserver of sorts.
• Whittlebit: www.whittlebit.com
  A search engine that learns through user interaction
• Kanzi: http://cematics.com/site.php/kanzi
  A neat little AI hack that helps webmasters do their job easier
• Uprizer: www.uprizer.com
  A "edge distribution network" that will optomize content distribution. It uses some Freenet Technology

Re:Post-Google Searching

[notsoanonymouscoward]

add netnose to your list!

Google Problems

[Ateryx]

Although probably bias as it is by msn, there was an excellent article about the faults of google in a past article

Unless I read the article incorrectly, this response-feedback-accuracy was the exact cause of the problem with google as shown by msn.

Just an observation...

Re:Google Problems

[hkmwbz]

If you had bothered to read the comments to that story, you would have seen that the article was fatally flawed and that the author completely failed to get his point across, and ended up looking like an idiot to millions. Just read some of the comments in the link you posted!

Here's an example.

Monopolistic myopia

[Anonymous Coward]

Yes, yes we should rely on a single source for searching for information. Any attempts to develop alternatives to Google should be ridiculed.

Stupid diversity and competition - who needs it!

Re:Monopolistic myopia

[Anonymous Coward]

I agree. Microsoft, for example, is trying to encroach upon Google's hard-earned territory by implementing its own "search engine." This "search engine" will be filled with ads, FUD, and other Microso$ft bullcrap.

I highly recommend that you use Google and shun all alternatives.

Re:Monopolistic myopia

[jdkane]

Microsoft's MSN Search seems to be a fair search engine. Enter the word "google" at MSN and it comes back with Google at the first item in the list, and even marks it as a "Top Pick"! Certainly no anti-competitiveness displayed there. Seems straight forward and honest to me.

You mean Microsoft would put in advertising like the other search engines put in advertising? Then it doesn't seem that unusual to do.

, and other Microso$ft bullcrap

I certainly hope they don't put a clippy mascot in the search engine results.

Pagerank cool

[MxTxL]

Page rank is cool, uses distributed data to improve search results. Definately AWESOME in the search engine world.

BUT i would also like to see the distributed concept applied to searching itself. Something like this idea, but having the engine return results on what were popular click-thrus for searches. From what i can tell (IANA Google Expert) Google isn't keeping click through data on search results (they are on the adwords, but that's different). By tracking click thru data and calculating how long a user stayed at a clicked result before hitting the back button or otherwise returning to google... good insights can be learned. Aggregate this over millions of users with billions of page views... wouldn't take too long to figure out what everyone wants to see for a particular search result. Combine all of that with improving your searches by what others are searching for... i think you are talking a powerful system.

Granted this whole idea may be liable to spamming and all of that... but that's not part of the concept yet. On the surface, it seems like a good idea.

NOTE: I know other engines track click thrus, but i don't think any of them do it for non-advertising purposes.... if it's purely to improve results then cool. If it's to show you better ads, not cool.

Re:Pagerank cool

[poot_rootbeer]

By tracking click thru data and calculating how long a user stayed at a clicked result before hitting the back button or otherwise returning to google... good insights can be learned.

True, but remember this: hitting the back button on a browser often doesn't generate a new HTTP request for the page you're returning to--normally the browser will just re-draw the cached version of the document that it already retrieved.

Thus, to the Google session tracker, it will look like you followed the link and never came back to the search results -- therefore, the link was a good result.

Re:Pagerank cool

[MxTxL]

I did over simplify there, the actual tracker would have to be smart enough to know that when a person clicked the next site after having backed up from the first one that they had gone back. There will be some loss there, but with aggregated results it should get lost in the statistics.

The whole thing is further complicated by people using tabbed browsing and multiple windows and such, but still in the base case, it's an interesting idea.

Totally unneeded.

[Kickasso]

Web pages are already rated -- by other web pages. Ever noticed these blue underlined chunks of text? They are called links. Each link is a rating that says "Lookie here, I liked it and you might too!" And somebody already uses this rating system in a search engine. Bonus points for correctly guessing who.

Re:Totally unneeded.

[martyn s]

Not everyone has their own web page. And the stuff on web pages, things that people publish, don't always reflect what users are looking for. This is really like page rank, just taking a much bigger sample (more users, not just people who have web pages). Plus it will probably balance out the disproportionate amount of power of incestuous links (scientologists, bloggers).

Re:I BELIEVE IN GOD

[Anonymous Coward]

birds can never understand how to make a pizza

But apparently monkeys CAN be taught how to post messages on the internet...

Not prone to abuse

[blchrist]

I don't understand how this system can be abused. From the post: WhittleBit will use your feedback to determine which keywords should be added or removed from your search, then you can search again to get more accurate results.

People are not changing how the search engine ranks the results for other people, it is just slightly modifying your query to produce more precise results. How can that be abused to make trash sites show up with rank 1?

Re:I BELIEVE IN GOD

[Anonymous Coward]

This view has been successfully disputed for centuries on the basis of something known as ASTRONOMY. Please, sir, wake the fuck up.

Ok, back up - kinda

[Sanity]

Ok, it is back up after I killed the "whittling" engine - feel free to play with the UI, but it won't do anything intelligent.

This was more intended as a proof of concept - rather than an all-out replacement for Google. I was frustrated with the way that Google works really well if you are looking for something easily defined and-or well known, but trying to find something obscure that was "masked" by more popular sites with similar keywards could be a real PITA. Whittlebit is designed to automate the manual process of trying to refine your keyword choice to get the search results you want.

Re:Ok, back up - kinda

[Suidae]

Pardon my ignorance, but how is this different than entering my search term, identifying the topics that I don't want to see, then excluding them and searching again? I do this frequenly with google, like if I am searching for 'mushrooms', but I'm not interested in tripping, I can add '-magic -psilocybe' or '+morel' or whatever.

Does this do some kind of bayesian-like filtering on the pages to get me a better match than I'm likely to come up with by using keywords?

Re:Ok, back up - kinda

[Sanity]

Pardon my ignorance, but how is this different than entering my search term, identifying the topics that I don't want to see, then excluding them and searching again? I do this frequenly with google, like if I am searching for 'mushrooms', but I'm not interested in tripping, I can add '-magic -psilocybe' or '+morel' or whatever.

It is different in that it automates the process of identifying which keywords should be excluded, rather than relying on the user to figure it out for themselves (which is often non-trivial).

Re:I BELIEVE IN GOD

[Anonymous Coward]

Son, you're not funny. I'm quite sure my IQ is a lot higher than yours. Sorry.

Google do this

[Richard5mith]

I'm sure I've seen Google do this. I've occasionally seen that links I click on in Google search results get forwarded through another Google URL which is no doubt tracking what I'm clicking on.

Like a lot of Google features they're testing though, it's very much random and it's been a month since I've seen it.

Re:Google do this

[sageFool]

Oh my god google click tracks you? No way!

Every search engine does this as far as I'm aware, but knowing what links people clicked on in a result set still doesn't help you much in figuring out if the results were any good or not. The user can click the result, in 2 seconds decide it is crap, go back, click the next one, repeat. So you might think you could just say 'If the user comes back and clicks another link from the same result page in 2 seconds then the first link was crap' but that doesn't really work since a user could click a link see the answer they were looking for in under 2 seconds and go back and click another link to see if the answer is the same elsewhere. Or they can spend like 5 minutes before they decide the link is crap. And users all have different ideas about what is good and not good about results for the same keyword or phrase. For that matter dealing with phrases starts to make things even more of a mess since a large % of all search phrases are very unique, so it requires lots of data and resources to apply statistical methods and try to do something reasonable on the data, and of course then if you want to do it on a per user basis you end up multipling the resources you need even more.

Not that I'm saying useful things can't be done, but it is not a trivial problem to be solved. Of course I know a number of people at google who are really bright phd types (since that appears to be all they hire ;) so I'm sure they are thinking about all these things as they are very engineering driven. Yay search engines!

Re:Google do this

[Anonymous Coward]

Not only every search engine, just about every website too ... apache logs are default for a start and just about every website has a counter, so at least *some* form of 'tracking' is nearly always going on .. The point is though .. who cares? OH NO! They can send me better targetted advertising that I'm more likely to be interested in! it's an extreme invasion of my privacy! , I say again .. who fuckin' cares?

Re:Google do this

[Miles]

When I use search engines, I look at the bit of context to see if it might be interesting. I think you could then use that context as feedback. It wouldn't be as good as actual user feedback, but better than nothing, I think.

Re:"Free Search" has no place in the commercial we

[InSinU8]

I'm not sure why a search engine would need to "represent an even distribution of wealth."

Any system like this with such a clear need for top ranking is going to invite abuse. You can find that sort of abuse in the local phone book if you'd like to take a look. Notice all the "(A|AA|AAA|AAA) company" listings. What happens with search engine spamming is similar.

Who is it you are suggesting should pay, the users in a Lexis-Nexis style model or the submitters in a Yellow Pages fashion?

Anybody remember DirectHit?

[Anonymous Coward]

It had a similar user-ranked search engine... Going to the website now redirects to Teoma. It had a similar green color scheme to the old HotBot site (which has now been bought out by lycos).

Boy, thinking about all these old search engines really brings back memories. These young people today - they've been spoiled by Google. Back in my day, everybody used a combination of search engines, and even then nothing worked right.

WELL?? WHERE'S THE EXPLANATION???

[Anonymous Coward]

Where is the explanation??? Can not explain it???

Does NOT work in practice

[veldmon]

Mr. Ian Clarke is truly a genius in his field of distributed systems. Is there much room for criticism of some actions he has taken though? I think so.

All products must reach a critical mass at some point. Otherwise, quality products will go into oblivion, e.g. BeOS. The fundamental flaw with Mr Clarke's work is that it was motivated by a desire to make digital viewing of child pornography undetectable by law enforcement. These kind of products will never reach a critical mass, and in turn never flourish in the mainstream market.

Re:Does NOT work in practice

[daveashcroft]

Speaking as someone who (used) to know Ian personally, what you are saying is (at the best) deluded and (at the worst) plainly libelous. You have missed the whole point of freenet. I suggest you carry out some appropriate form of redemption to the gene pool by "experimenting" with tabun.

Re:I BELIEVE IN GOD

[Anonymous Coward]

LEARN TO SHOW SOME RESPECT you illiterate OAF!

Congratulations, you've just won the Coherence Award.

Re:I BELIEVE IN GOD

[Anonymous Coward]

Hey asshat don't you see this thread is about JOKES on the subject of God(and birds and pizza)? We don't need your pompous "I know it all" attitude here, go post on a thread that is not offtopic and leave us alone. Also, YHBT, YHL, HAND.

I don't know about this

[ChiefArcher]

I think people will start making their websites look better.. and then make other ones look bad (like it's been said in here).

What if i get a list of proxys.. write a program and click on each of the links and rate all of them..
It's easy as that... I don't think it'll work.

All the porn and viagra sites will be #1

Chiefarcher

Re:I don't know about this

[Anonymous Coward]

All the porn and viagra sites will be #1

And this is different from google how?

Re:I BELIEVE IN GOD

[afex]

heh, its pretty funny that you called me 'tough guy' when my post included 'final fantasy'

Re:ATHEISTS, EXPLAIN ME THIS!

[Anonymous Coward]

Wow! That is pretty amazing. I can't explain this picture. (and I'm not atheist)

In other news...

[ikkonoishi]

Server declares "Nobody loves me" before crashing and taking down the search engine which allowed users to rank its results.

Experts believe this was due to repeated thumbs down given to its site within its own results.

Re:I BELIEVE IN GOD

[Anonymous Coward]

So, monkeys with high IQ can post on the internet.. boo-hoo, sue me for not being as pointillous as your snobby self.

Abuse

[Ed Avis]

So how do you deal with trolls and spammers who will vote up or vote down sites for partisan reasons? Or ignoring that, what about straightforward differences of opinion? (The world may be polarized 50/50 between those who think 'firebird' refers to a database and those who think it is a web browser - at least among the geekier-than-average WhittleBit users.)

Anonymous feedback won't scale well to the big bad Internet; some kind of login and network of trust is needed.

Re:Abuse

[sageFool]

And those that think it is a bitchin' muscle car.

THough I think they plan on making the ratings only apply to the individual leaving them, which prevents abuse, but probably is an interesting challenge to prevent it from using silly resources.

There it is

[Anonymous Coward]

Photoshop. This Morrison picture is all over the place and of course, Jimmy's ghost would take that same pose just for fun down at the cemetary.

If such crap is the cornerstone of your worldview, I really, REALLY feel sorry for you.

Re:There it is

[Anonymous Coward]

Send this guy an email and ask about it! He has his address there. This guy is a well known journalist in USA. And it was Tom Petty who took the picture. I'm quite sure it's not fake. That would ruin your career.

Re:I BELIEVE IN GOD

[Anonymous Coward]

Don't choke on your muffins kid! Looking at your sig all I can wonder is how old are you? 13? 15?

Re:Ack! Do you know what you're doing?

[xyzzy]

You're missing the point. The system isn't watching user actions while searching to fine tune OTHER user's results, but to fine tune THAT user's results.

While you can certainly claim that one user's actions MIGHT indicate relevance for another user's queries, it's certainly true that if a user gives you a clue that the document you have returned is irrelevant, it must be irrelevant.

Google still has human ranking...

[arhca]

It has yes/no voting options on its toolbar.

Re:I BELIEVE IN GOD

[Anonymous Coward]

15 AND A HALF! And don't you think that just because I'm younger than you I can't think! I got my opinions, and one of them is that you suck.

Re:"Free Search" has no place in the commercial we

[Boss, Pointy Haired]

I'm not sure why a search engine would need to "represent an even distribution of wealth."

Of course there is no requirement on a search engine to represent an even distribution of wealth, but it is in the SE's own interest to if it does not want to become the spam-fest that is the commercial Google.

When there are thousands of companies providing $service, why should serch engines direct the overwhelming majority of traffic to the one site that happens to fit their algorithmic opinion the best.

Any system like this with such a clear need for top ranking is going to invite abuse.

Exactly, that is why I am saying that free search has no place in the commercial Internet.

Google has this too

[acm]

If you install the google toolbar you can vote for or against pages on an individual basis.

acm

Re:Google has this too

[Henk Poley]

And, will this affect your personal search results in the future?

Commercial sites overflooding search engines

[DRWHOISME]

Something needs to be done to seperate stories,informative article useful for research and education from the crass commercial websites that are like SPAM on all search engines. Some sort of separation needed. Do something about that and i will be happier. Just type in anything about money or business on all the search engines and you will be flooded with irrelevant links.

Re:Commercial sites overflooding search engines

[xyvimur]

That's true - but do You see any possibility of changing it? I mean real one that would work. I've we as the users get to rating the stuff - there would be bigger mess than in ordinary search engine - because everyones searching for something a bit different. Besides, as I mentioned somewhere earlier - how would we protect against artificial rating up of `crap'? Maybe some kind of `slashdot moderation' :) - but for search engine this may not work though.

Re:Ack! Do you know what you're doing?

[MikeyO]

>Imagine for a moment, a geek for hire, such as myself, writing a PERL script and deploying it on several servers nationwide. It uses LWP::UserAgent and spoofs...

You need to read the description again:

will use your feedback to determine which keywords should be added or removed from your search, then you can search again to get more accurate results.

This does not imply that the results of your feedback would affect somebody elses search.

since this got modded up to a 5, i'd guess that the moderators should be reading the description again too.....

commercial search?

[commodoresloat]

What do you mean by "commercial search"? Search engines are for information. Information can be used for commercial or noncommercial purposes. "Advertising" and "commercial information" are two different things. If you're just talking about searching for products or commercial information, "free search" engines work fine; the information is still findable. If you're looking for specific commercial information, go to the company's website and search there. But I don't agree with what you seem to be saying, which is that search engines should be advertising disguised as a reference tool. Screw that. If I want advertising, I can watch TV. Or set up a commercial search engine that announces itself as nothing more than an advertising engine, and you can sell product position all you want. But when I want information -- even information about commercial products -- I want that information filtered by relevance, not by who paid the most money to deliver the information to me. If I want to look up "safe cars" on the web, I don't just want ads from Chrysler; I want information from different perspectives. There is a huge difference between information relevant to commerce and information that is bought and paid for.

Re:commercial search?

[Boss, Pointy Haired]

But I don't agree with what you seem to be saying, which is that search engines should be advertising disguised as a reference tool.

Google is a reference tool through which people are advertising, and as an advertising medium, it is not good.

That's the problem.

By commercial search, I mean a search for products and services, such as "web hosting". There are thousands of companies providing "web hosting", but you go to Google, and the same company is #1 every time.

That isn't right.

Or set up a commercial search engine that announces itself as nothing more than an advertising engine

Exactly - this is the way it needs to be. Google may even be laying the groundwork with Froogle, but there's still a long way to go.

Re:commercial search?

[Anonymous Coward]

Google is a reference tool through which people are advertising, and as an advertising medium, it is not good.

Thank GOD for that.

By commercial search, I mean a search for products and services, such as "web hosting". There are thousands of companies providing "web hosting", but you go to Google, and the same company is #1 every time.

That isn't right.

Yes it is. If the most relevant web hosting company is the first one, then Google works as it should.

Re:commercial search?

[esconsult1]

Some time ago after a heated argument, we decided that Google does not provide commercial search.

The most perfect commercial search engines belong to Ebay.com followed by Amazon.com. As the previous poster said, there is no way to quickly evaluate commercial results in Google... Froogle maybe in the future might offer decent commercial search, but Google won't because of Pagerank, which is fine for searching how to format a Linux hard disk, but not fine for searching where to buy Linux hard drives.

I know, I know, Adwords are there, but If I really want to compare and buy stuff, I'm heading off to Ebay or Circuit City (or maybe BestBuy with their broken Javascript site).

Google's great and all, heck, I even am a customer of theirs, but for commercial search, then leave a lot to be desired.

Masochist and Sadist

[handy_vandal]

Yea...until you get a system that's into S&M.

A little reverse psychology will fix perverse, disobedient systems:

"The masochist says 'Hurt me' ... and the sadist says 'No' ..."

Better for limited document sets

[realfake]

While the idea has plenty of problems for use on a general web search engine, it could work very well to tune results on a site's internal search engine, where the user has no vested interest in one result coming up higher than the others, the user only wants good results.

It might also have potential, even if the thumbs up/thumbs down are only shown to trusted users. One of the enduring problems in tuning search engines is that the people who build the search engine aren't the people who know the content best. Getting the content people some way to say "yes, this item should come up higher for this term" is a powerful idea, IMO.

Re:I BELIEVE IN GOD

[Anonymous Coward]

Well I'm 47 and I got the same opinion anyway. This troll's paternalizing is downright ridiculous but eh, isn't that what religious establishment is all about ?

Netnose!

[notsoanonymouscoward]

Not trying to steal the show too much from whittlebit, but theres another new search engine recently released. Netnose lets the users decide which keywords a web page should be listed under. The search results also include handy identifiers about the page content like whether it has popups, or contains adult material (as decided by the raters).

Re:Netnose!

[.gc]

I visit NetNose every now and then and play with the rating interface. Its cool and it seems like they've put a lot of thought into making it easy to use - I think it has a lot of potential.

Re:Netnose!

[vldmr_krn]

Netnose lets the users decide which keywords a web page should be listed under.

Nooo! Don't do it! It's powered by PEOPLE!

Google phoning home...

[presroi]

Could improved interactivity be the next big search engine advancement after Pagerank?"

Well, actually, Google does receive feedback. Once in I while, google changes its result page in a way alexa is doing every time:

You don't get a url to the result back but rather a pointer in a way like www.google.com/result?target=realurl.

I'm sorry that I can't provide you a real url but I'm confident that someone in this /.-crowd can help me out. Thank you in advance.

Finally...

[Anonymous Coward]

... I can find porn the every one else sees and not end up in popup hell :)

Re:Ack! Do you know what you're doing?

[Christianfreak]

Well on a per-client basis, his point about the embedded spy-ware would certainly be valid.

Picture this

[Anonymous Coward]

- Hello sir, you're the one who took that Jim Morrison picture in Paris ?
- Yeah, pretty weird isn't it ?
- Well, I was wondering if you didn't "help it" a little...
- What do you mean ?
- Hmm... We can obviously see Morrison take this pose all over the Net, and we can see the picture messing up on the alpha and...
- I tell you it's the real thing man, believe me.
- It doesn't make sense sir. The perspectives are wrong and the picture has clearly been cut out of some place else.
- I'm telling you it's real. Believe me.

Yeah, of course I'll write to this guy. How constructive.

Re:Picture this

[Anonymous Coward]

Well.. How about:

- Send me a copy of the negatives.

You hopeless moron.

Re:Picture this

[Anonymous Coward]

Because you can't modify negatives, of course.
Oh wait, YOU CAN.

You don't know shit about photography ; if you did, you'd see pretty quickly that this picture is a fake.

Open for exploitation

[arth1]

I can see vendors writing scripts that will at random times access the search engine with searches related to their product, and automatically give "thumbs down" to high ranked results not affiliated with their own products, and thumbs up for their own pages.

For pr0n, this would of course happen an order of a magnitude more often, starting two days BEFORE the search engine launches.

Regards,
--
*Art

Abuse

[sageFool]

Seems totally open to abuse, and there seem like their are issues with people not rating results and keeping the statistics meaningful. If they can get something up for doing ratings and figuring out if a user thinks a result is 'good' or 'bad' that is easy for the user to use, isn't abuseable, and has some kind of statistical validity I will be impressed, but I think it is much harder to do than most people think. Yar!

Re:I BELIEVE IN GOD

[afex]

well actually i'm 19...but thanks for the warning....the other day a lemon poppyseed muffin did give me somewhat of a problem...i'll be more careful in the future. maybe its cause i'm not praying enough.

General Lack of intelligence by posters

[gurps_npc]

While posters seem to have high programming skills, most people here are assuming that your thumbs up/down affect other people's searches. That is not the case.

This is not the first time I have seen people make similar dumb mistakes.

I think it has to do with not reading the actual article/sample. /.ers tend to skim a lot.

Re:General Lack of intelligence by posters

[sageFool]

And the articles linked to by /. tend not to be up for very long. So one has to make assumptions. The assumption that thumbs/up down affects everyone seems vaguely reasonable because of the resources that one would need to track the ratings of ever user of the search engine on an individual basis. It is in fact the assumption I made, so you just better shoot me in the head for being such a dumbass.

Re:General Lack of intelligence by posters

[gurps_npc]

While the real article was not up for very long, there were several links to valid info about the product. If you clicked on anything related to it, you quickly found out that your choices did not affect anyone but you.

More importantly, if you did not get any of the details, then people should keep their mouths shut. What would you think about someone that gave a review of a movie, after they saw the "previews", without ever seeing the movie?

Finally, this fact SHOULD have been obvious to anyone that thought about the issue for a second. Instead they assumed the product was done stupidly, without bothering to think, hey maybe the person that did this is not incredibally stupid, and has considered this issue. The people that posted using this erroneous views did not read any of the various links that were posted, and did not bother to spend 5 seconds thinking things through.

Re:General Lack of intelligence by posters

[Anonymous Coward]

If you read the synopsis of the article it clearly states that the voting will only whittle your result set, not anyone elses. It is amazing how many posts are about abuse when this system makes it very difficult for abuse to occur.

Re:General Lack of intelligence by posters

[Anonymous Coward]

Actually your voting will have no affect whatsoever since the results are from Google and the whittling is nothing more than -keyword added to your original search. Seach on whittle copy to Google and you get same results. Thumb down a entry and copy the exluded words to Google and place a - in front of each along with original search and whalla same results. This site is a Google front end interface.

Re:I BELIEVE IN GOD

[Anonymous Coward]

...while making assumptions about someone's IQ after one sentence is a sign of great intelligence, of course. Thanks for the tip mate.

This is not new...

[Japanese Dad]

Variations of these re-ranking schemes have been around since the 1970's. This sort of scheme is also easy to do implicitly by tracking which results have been clicked on and then using the keywords in those clicked-on results to improve the original search terms. Hotbot.com, among others, used to do this.

Re:This is not new...

[bitkid]

it's not new indeed. Another example of the same idea is to record the URLs the user actualy visits, which WhittleBit seems to do as well. A search engine that's learning from these click-through alone is Striver. It's based on paper by Thorsten Joachims dealing with Support Vector Machine based ranking. Pretty cool stuff, but (to my knowledge) the current algorithms don't scale to the size of a full-scale web search engine. Also IIRC the explicit user feedback didn't work too well in some experiments conducted by information retrieval researchers (I don't have a link handy for this one)...

Re:I BELIEVE IN GOD

[Anonymous Coward]

Don't go ape pal! HarHarHar!

Re:Ack! Do you know what you're doing?

[Cyno]

I guess to get around this we kinda need to create a list of "friends" or people who don't abuse the service. You know, honest people who care about things other than money.

Another way to do this is for each user check to see how they are voting compared to the other votes for that site. If they are consitently opposite the public reaction then they are most likely some sort of troll. Also limiting the number of times someone can submit feedback could cut down on abuses. Thresholds 'n stuff.

It's called "Relevance Feedback"

[gbnewby]

In the academic field of information retrieval, this is called "relevance feedback." It's a part of many information retrieval (IR) algorithms, some of which can happen automatically (i.e., unsupervised). There is also overlap with the fields of machine learning and even Bayesian processes (see today's other /. story about spam filters -- spam filtering is actually the same problem, conceptually, as search engines try to solve).

In Yahoo and other search engines (but not Google, that I've seen), you often get a "click-through" that goes to their system before transparently redirecting to the actual URL you clicked. This is relevance feedback. It's true that the system can't determine whether you LIKED the site (aka, whether it was "relevant"), but at least it's some sort of feedback the system can use to tune.

The other most familiar type of system I can think of is Alexa (now owned by Amazon.com, and the brainchild of the Internet Archive's Brewster Kahle). With Alexa, they could count not just that you visited a site, but how long you spent and where else you went. This is at least part of the basis for Amazon's recommendation system for books and other geegaws they sell.

Can this work in a search engine? Yes, certainly. Does it mean that a search engine that implements relevance feedback will instantly be better than Google? Definitely not! There are many other things (about 20, from what I've heard) that go in to the ranking system that Google uses...Pagerank is one of them, but there are many other factors (such as term frequency, document HTML structure, etc.). Some these, notably Pagerank, work poorly on relatively small collections (in the TREC conference, people have almost never found that Pagerank, HITS or similar algorithmns improve performance with "only" a few tens of GB of Web documents -- a few million pages).

Wanna know more about information retrieval? The TREC page above is very good for state-of-the-art research reports (see the Publications area -- it's all online and free). More general texts are mostly in libraries, but one good one online is Managing Gigabytes, which covers the IR aspects thoroughly and also has lots of ideas about how to use compression in an IR system (something that I'm curious whether Google & others do).

Re:It's called "Relevance Feedback"

[bailster]

--"With Alexa, they could count not just that you visited a site, but how long you spent and where else you went. This is at least part of the basis for Amazon's recommendation system for books and other geegaws they sell."--

Interesting. So if I surf on-and-off all day while at the office and periodically right-click several links at a time -- but only read the resulting pages much later when the boss isn't looking -- do all these windows that have been sitting open for hours count as "positive feedback"?

Maybe *that* explains why I get such crappy recommendations on Amazon!

So why *isn't* this being done?

[siskbc]

No offense but:

In general, statements like that are used by people who haven't actually thought through the algorithm in detail, or who don't have good knowledge of algorithmic theory.

None taken. Put it this way - I could write it in matlab, and I could write it pretty bad in C++. However, I'm not familiar with google's code, and wouldn't be able to integrate it into that. But I could write a version of it, just not as it would need to be, final form. In other words, I'm very familiar with the algorithms involved, that's definitely not the problem. I do work on problems similar to this in grad school - the source of the data is completely different, but the same tools can be applied.

In specific, your suggestion sounds excellent. Sufficiently excellent that I would be very surprised if Google, with their famously large R&D division, didn't have some very smart people thinking about it or something similar.

Thanks, and I agree - if they're not doing this, they should be/have been. What I outlined would be reasonably accomplished through new applications of existing decision theory algorithms.

Thinking about it briefly the first couple aproaches I come up with wind up being factorial time. Plus there is a lot of fuziness as far as how to promote Fiona Apple links but not just lousy Apple Computer ones, not to mention search terms where the "families" of hits are less distinct than for Apple.

it's not as fuzzy as you'd think, and I think this could be done with less computational overhead than you'd initially believe. Basically, what we have is a classic supervised pattern classification algorithm, where the two classes are "useful" and "not useful." At the point where you tell it the groupings, then it's just a matter of determining what characteristics are common among the groups. You'd have to reduce the results to more ordinal characteristics, but this would be a solution similar to how mozilla translates emails into vectors of charactersitics for their Bayesian mail filters.

Most of this could be done starting with, say, a few hundred results or so per search. Arranging into categories from here would be fairly trivial, at which point those categories would be presented to the user. The user could then update the relationships as they are determined by the computer, and resubmit.

Of course, the more samples you use, the more overhead. Also, the more descriptors/features/parameters, the mroe overhead. Using one way of doing it, the problem would be linear with samples, and O(N^3) with features (due to a matrix inversion). Not all that bad, particularly when the number of features can be capped, and does not grow (necessarily) with samples.

Re:So why *isn't* this being done?

[Anonymous Coward]

Somebody is doing it.

Nice TROLL!

[spun]

You really got some folks worked up with that one. As for the rest of you, how could you fall for such an obvious troll?

Re:"Free Search" has no place in the commercial we

[DarkZero]

I'm not sure why a search engine would need to "represent an even distribution of wealth."

I know exactly why: because he's joking. Look at his user name.

Obviously, they need some sort of Meta Moderation

[spun]

You know, let random folks rate the ratings that other people give, something like that. I know it's a radical idea, but I bet we could come up with something if we tried.

I take that back...

[Doesn't_Comment_Code]

I just checked the source code on Google.com and they DO have javascript on their main page. So I eat my words from that last post.

If I get time I'll look through all of it and see what it does.

Re:Obviously, they need some sort of Meta Moderati

[The Bungi]

That would be a good idea, but how do you keep metamod from being abused as well? Hmmm... a /. styled random system would probably help.

sounds like an idea to me.

[DRWHOISME]

good points.

I posted the same thing

[DRWHOISME]

only a little later than you. Try and find anything to do with money or business or anything that involves entreprenuership and you will be disappointed.

Mod parent up

[Henk Poley]

Since the majority of the poster doesn't seem to understand this.

The problem with human interaction is...

[ChaseTec]

...there would be human interaction. Wasn't the whole point of page rank so that results couldn't be manipulated so easy? Yeah I know there is that link exchange stuff that google had to deal with but for the most part it works. As soon as we allow humans to voting someone will start to screw with the system.

Re:The problem with human interaction is...

[Anonymous Coward]

This is not true. Netnose has already set up a system with human rating and solved the problem of abusive voting.

Now fully operational again

[Sanity]

The hitrate has died down enough that I think it might be able to handle it - I switched it back to full-functionality about 5 minutes ago and it seems to be coping - lets hope this continues (it is downhill from here - right?).

Doesn't have to be useful for everything

[siskbc]

So, if Google's tool could also be used to identify the different meanings that each word has, then maybe they could give you a few links for each meaning.

That's very much the idea.

Of course, many search items might not be this easy to categorize.

That's true, but at point the most successful solution would be what google does now. In other words, if there isn't any clear substructure to the organization of results, just return the most relevant.

We haven't yet proven that your algorithm is easy

Obviously, since I don't have access to google's database of webcrawls to test it. However, I will say that the problem reduces to pattern classification tasks that are reasonably well characterized. I have a pretty good background in that, and am familiar with techniques that work very well with problems similar to this one. I'm actually fairly confident this would work.

and we also haven't proven that it would be useful for many types of searches.

As I said, it doesn't have to be useful for all searches. It simply has to be useful for the categories of searches at which google now fails. The "apple" search is one of the prime examples of google's failure, as it's pretty damned good at everything else.

Please, please read the article

[p3d0]

Your ranking only affects your own search results.

Knee-jerk

[p3d0]

I have never before seen so many ignorant replies rated 5. Please refer to this.

Read the article

[p3d0]

Your feedback only affects YOUR search results.

NetNose

[hyeh]

I've seen this already on a search engine called NetNose. Nothing original.

Pr0n search.

[ratfynk]

This will greatly enhance your search for free pr0n.
Isn't the internet great!

That's about it

[siskbc]

I checked it out, and gave it the "apple" test. Didn't do bad, but I think it could be done better. I might talk to the guys, thanks for the link.

Partial auto-induction

[sacrilicious]

I wonder how much of this information a search engine like Google could get for free by paying attention to these factors:

• a) which results actually get loaded by a user
• b) the gap in time between successive result load
• c) the last result a user loads

I can imagine useful interpretations of each factor such as (a) links that get loaded were more useful than those that were skipped, (b) longer times between loads might mean richer relevance to user's search, and (c) user definitively found what they were looking for upon viewing the result. Granted there are common load patterns that would throw these interpretations askew, but the very same thing could be said of the information that google currently uses to rank links, and there is still enough statistically valuable data there to make them a great search engine.

Google doesn't give me what I want!

[zerocool^]

This could be useful for those cases where Google just refuses to return the search results you want.

You mean, like, where I search google for "Porno and Snuff Films", and I honest to god only want results pertaining to the song by the band "The Lawrence Arms".

Heh, wish they could do that on kazaa.

~Will

Regular expressions...

[Evil Attraction]

...is the only thing Google is missing. At least that I know of. I know it won't happen anytime soon, though, as it would kneel the Google servers in no time.

But how about enabling regular expressions on your search results, ie. you do a normal search first, and then do a regular expression search to refine the results from your first search? That would've been something, and I guess it wouldn't have required that much CPU power (as compared to allowing regular expressions in the first place).

I'm doing quite a bit genealogical research, and regular expressions would have been nice when search for people and dates (ie. years) related to those people.

--
Evil Attraction

Ian Clarke Is Leaving the US

[morcheeba]

Ian Clarke announced in this slashdot comment that he's leaving the US. Here's the story at boing boing.

Google and Whittling

[gnugnugnu]

Getting user feedback is easy.
Getting useful user feedback of the kind you actually want is hard.

This would be ideal for something like the Google image search, where there are a large amount of broken links and the required user feedback is fairly simple: Broken Link.
Based on that user feedback Google could check the page to verify if the link is in fact broken.
The cornerstone of any rating system like this is trust. While it might be useful for improving your own search, I really dont see how this kind of system could be shared among a community and scale properly. I cant imagine it would hold up for very long without a very complicated trust metric.

Google and others already retrieve user feedback

[harmonica]

There is no real interactivity, but Google sometimes encodes links in search result pages in a way so that it can log which links are actually clicked on. Simply by replacing http://site.com/page.html with http://google.com/query+terms+encoded+here/referer /site.com/page.html (or something like that, you get the idea - Google gets the hit, logs it and then forwards the requesting client to the real page).

Other search engines do this all the time.

However, I remember one Google employee saying in an interview that this information is not integrated in the actual search engine - yet.

Already done for Google

[_TheShadow_]

Tibianna is a system that uses machine learning to refine a Google search. See http://www.thestaticvoid.net/portfolio/p_tibianna. html

How to emulate WhittleBit in your own applications

[OllyBetts]

In the academic field of information retrieval, this is called "relevance feedback."

Indeed. And if you want to experiment with relevance feedback, take a look at the Xapian Project for a highly scalable GPLed implementation of the Probabilistic Information Retrieval Model (which is derived from Bayes Theorem, the basis of all those Bayesian spam filters).

The only search demo up at the moment is the search over the site itself, which doesn't show off the relevance feedback especially well - the pages on the site cover a rather narrow topic area, which is fine for searching the site, but less good as a demo. But if you try it you'll see it suggests various related words and you can click on these to add them to the search. Additionally you can click a checkbox next to a hit to indicate that it's relevant - check a few and hit search and the suggested words will be based on those documents you liked.

Where's the mirror?

[frostman]

Warning: fsockopen(): unable to connect to 127.0.0.1:9182 (Connection refused) in /home/ian/whittlebit.com/wqserver.php on line 13

Connection to WQServer failed

/.ed already...

*sigh* maybe I can test it later.

As an aside, I find it a bit annoying that it uses frames and that the query input is readonly when you have (or would have had) a results page, so you can't enter new terms then click "New Search." Since it uses Javascript anyway, that would have been easy enough.

Slashdotted?

[tim.kerby]

I searched for electronics to see how many hits I got and look what comes back:

"Warning: fsockopen(): unable to connect to 127.0.0.1:9182 (Connection refused) in /home/ian/whittlebit.com/wqserver.php on line 13
Connection to WQServer failed"

Yet the page itself seems to load really quickly

taxonomy

[siskbc]

The reason why Google doesn't offer anything similar might be because Northern Light has patented their subject classification and taxonomy.

For what it's worth, the way I'd do it would *not* require a subject/keyword taxonomy. To me, that's one of Google's strengths - they don't attempt to evaluate content ahead of time. I wouldn't either - in fact, I think you *have* to develop the substructure/classification relationships based on the results of the search. Otherwise, all you have is a fancy card-catalog system.

I read their patent (5,924,090), and it was fairly badly written (even compared to other patents!). I'm not sureif my method would be covered, as I wouldn't even attempt to use a taxonomy, as I think it would fail based on its very nature. It does seem as if they're trying to "0wnz0r" the entire idea of classifying results, which seems damned broad, and I think people were doing this far before July 2001 when they filed.

The whole taxonomy patent thing is weird, though. As I said, I've seen *numerous* websites offer up pre-designated categories based on search results (Yahoo, etc). Do all these companies license Northern Lights' technology, I wonder? If not, then the patent must be more specific than it initially seems.

As for ease of use, you're right - this wouldn't necessarily be the way you want the search engine to always work. To me, it would be like another version of "advanced search." You don't always want to use it, but you definitely want it there.

Slashdot Moderation + Google

[A55M0NKEY]

Google finds what you want pretty well as it is. If there are ambiguities, you can use the minus character. EG: someone wants mythology about Morpheis the god of dreams they could type: Morpheus -Matrix -filesharing dreams and get what they want. If you let people rate the results of searches for relavance you will get people looking for greek mythology, people looking for Matrix stuff and people looking for the filesharing program all rating the results differently causing a war that damages the quality of the results. They are all relevant to the word Morpheus.

What google really needs is a Yahoo email style 'This is spam' button. People love to punish so they will press that button if a site is obnoxious enough to piss them off.

Protecting that button from abuse would be a challenge, but I do not think impossible. Slashdot's moderation system seems to work well.

Here is my idea:

Random mojo points ( mojo = credibility ) is distributed to a few thousand random users every day.

These users rate sites as spam like everyone else, but unlike slashdot moderators, they do not know they have been assigned mojo. Nobody knows how much mojo they have.

People can use up to half the mojo they have each day. A person given one point of mojo who rates 6 sites as spam in a day has given each site 1/12 of a spam point, and will have access to 1/2 of his remaining 1/2 of a mojo point = 1/4 mojo point the next day. This is so that the person will be encouraged to rate spam continuously, not just when they think they might have mojo. ( they aren't supposed to know they have mojo anyway )

If the person rates only one site as spam the first day, then that site gets 1/2 a spam point. The first person to rate that site as spam gets 1/4 of a mojo point for agreeing with the first person. He can use up to half that mojo tomorrow. The next person to rate that site as spam, gets nothing for agreeing with the first person, but half of the spam points the second person assigned to the site. The site accumulates spampoints, but mojo is like an olympic torch that grows half a s bright each time it is passed. The analogy holds perfectly if each person rates only one site a day, but if more than one site gets rated by each person, then it is more like a torch that gets dimmed by half and then divided up and passed to multiple runners. The total spampoints assigned to different sites as a result of giving one mojo-point to a random user is 1 + 1/2 + 1/4 + 1/8 + .. + 1/2^n = 2 mojo points.

Because each randomly assigned mojo-point results in 2 mojo points being assigned, forgiveness formulas can phase out 2 spampoints for every mojo-point given out randomly.

Slashdot uses metamoderation to moderate moderators. This prevents moderators from modding up GNAA and WIPO posts. My idea uses agreement to distribute mojo-points instead of metamoderation.

If you assign too many mojo-points, then you open the door for scripts that create accounts that sit and wait for mojo-points from heaven and then use those ( relatively powerful ) first hand points to attack a competitor's site's google listing.

You want to make these heaven-sent points rare, so that it is not worth the trouble of the raterbot kiddies. Mojo points should be rare enough that the number of accounts you would have to have running as scripts to get a decent number of points would make you an obvious spamraterbot purveyor, and your site could be blocked. Most times this would be unneccessary as they would be drowned out by people with legitimate accounts. It would be impossible to create scripts that gleaned mojo by rating known spam as such since known spam would not appear in google listings and so would not have new spampoints ready to be aggreed with. Scripts trading exponentially decaying mojo with each other on lists of known spam sites is harmless. If the scripts decided to rate random sites as spam, then they would not get any mojo for it

I just discovered

[Anonymous Coward]

Uraninite grows around here. I found this guy who tells how to purify it w/ cheap stuff from the HW store. You could just go outside pick up the pitchblende, purify it and start yerself a lil' atomic pile. Sort of like a compost heap that you don't need to turn with a pitchfork. Radioactive decay don't need no air and water! Yeah, and some dude says how you can get radium from it too. Man and this kid made a nuke reactor in his parent's flower shed. I think a uranium powered pile would be kewler. I think there's some barium in ammongst the radium in ore... It slows neutrons down so they can cause further decay. I mean, if you put some dynamite and a fuse under the pile then when it got real hot KAPOW!! Any idjut could do it. And you'd have to be an idjut to try! I mean with all that radiation you'd prolly die.. It's kewl how all ya gotta do is pretty much put a bunch of radioactive crap in a pile and then it gets EVEN MORE radioactive and nasty. dude....

TROLL! MOD PARENT DOWN!

[Daetrin]

This guy is a troll. A quick glance shows that more than half his recent posts have been modded as Troll or Flamebait, and examining the responses to the last few positively modded ones shows that they're just trolls that fooled the moderators.

Here in the lab we're doing some work on using the principles of thermodynamics in order to improve search engines. The second law of thermodynamics states that in a closed system ethalpy will alway increase, which is a lot like the disorder cause by sites spamming themselves to search engines.

Is this the same lab as "Here in the lab for instance, many of my colleagues have been releasing their scientific papers onto Kazaa instead of through more established journals such as 'new scientist'." ? What kind of messed up lab is this exactly anyways? Got a reference?
And why are you working on applying thermodynamics to search engines? I thought you were working on _particle_ physics? "The word processor is good, although somewhere it is set to autoreplace the word lepton with leprechaun which is proving most annoying as I write my paper on particle physics."

Second of all, it looks like you mispelled "enthalpy". Second of all, enthalpy refers to the energy in the system, entropy refers to the amount of disorder. When you're speaking of disorder increasing you're speaking of entropy, not enthalpy. In closed systems enthalpy decreases as entropy increases.

Third of all, if the systems are so similar that the knowledge of one can be applied to the other, how is that you think you've found a way around the second law of thermodynamics? Either the systems are disimilar enough that your "expertise" isn't going to help, or they're similar enough that you could apply the results to actual physics and have created a device to generate infinite power rather than futzing around with search engines.

Finally, the above rules are only true in closed systems, which search engines are not. There are people writing new web pages all the time, and new pages getting referenced, and old references being improved, so the second law of thermodynamics doesn't even apply.

You can't even get your terms right and you're trying to apply them to the wrong sitaution. I'm not even going to bother finding counter examples to the rest of your drivel.

Free rider problem - Re:Sounds Great...but

[bailster]

---who wants to wade through results and rank them? I came here to search!--

Reminds me of... "Who wants to wade through posts and MOD them? I came here to read!"

But seriously, wouldn't we all benefit in the long run from giving feedback to a search engine (provided the thing can be troll- and bot-proofed)? They'll never build a good AI to take over the world and obliterate life as we know it until we COOPERATE and slowly divulge the network of associations that make us human.

Now excuse me while I go find my tinfoil hat...

Patent pending

[Zopilote]

I did some research on a similar idea in 1999 and 2000 as part of my CS degree. It is currently going through the patent process right now. Don't worry, I'm a techie like most of you and don't intend to send out "cease-and-desist" letters to anyone with anything remotely resembling my process. My main motivation is to have something nice-looking on my resume, and perhaps to profit if I can sell the patent to a company at some point in the future. It was also very educational to go through the whole patent process on my own with no lawyers...