Slashdot Mirror
Search Engine Learns From User Feedback
An anonymous reader writes "Ian Clarke, founder of the Freenet project, has set up a web search engine that allows users to rate each of the search results it returns. WhittleBit will use your feedback to determine which keywords should be added or removed from your search, then you can search again to get more accurate results. This could be useful for those cases where Google just refuses to return the search results you want. Could improved interactivity be the next big search engine advancement after Pagerank?"
Could improved interactivity be the next big search engine advancement after Pagerank?"
.sig on Slashdot). I was unimpressed with the results the first time (there were 8 or so to work with) and limiting with the thumbs down was of little use when there were so few results.
.02
In short, no.
I have tried Whittebit before (a user had a link to it in his
I can't see google's superiority being challenged by this at all. What else would Whittebit offer me other than this "feature"? I didn't see anything else when I used it (and in fact, was rather annoyed by the fact that it remained at the top of the screen while reading the link I was sent to).
No thanks, just my worthless
I think something like what Kaltix is trying has a better chance of replacing Google. However I don't see that happening either. I just think Google will learn from the user based systems
As x approaches total apathy I couldn't care less.
no, i dont want to have to give feedback in a search, I just want to type keywords and find related results ...
I like the idea of interactive page rankings. I don't think it should be the one decisive ranking alogrithm. But human interaction is just what search engines need.
I do a lot with Google, and it leaves some to be desired. The goal of Google is to make the ranking of pages partly out of the hands of webmasters, so they can't just trick the spiders. And that has worked very well for Google (serves over 70% of internet searches). But all page ranks are very cold and calculated. Maybe that cold, calculated rank is a good place to start, and then it's time for human reviewers to fine tune the list.
By the way, Google has attempted to acheive this concept of human ranking by watching to see how long you stay at a page you clicked on. If they rank a page 1, and you click it, and immediately return to the search page, they penalize that page. So if even Google is trying the same abstract concept, it probably has a future on the web.
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
This is a great idea in concept, but the potential for abuse is incredibly high (if it's implement on a system that actually matters, like google).
Imagine for a moment, a geek for hire, such as myself, writing a PERL script and deploying it on several servers nationwide. It uses LWP::UserAgent and spoofs a few different versions on IE on Windows. It then run searches for hot keywords that my client wants to rank high on. Then it 'mods down' anything it isn't my client's product, and 'mods up' what is, or links to, my clients products.
Set the script to run several times a day at each location. Write some spyware that does so in the background of a shareware-app-for-hire (Kazaa?).
You see where I'm going with this? Protections would have to be in place.
Karma: Chameleon (mostly due to the fact that you come and go).
I think this is for whittling down a person's individual searches. My preferences when I'm searching for something about rj45 plugs won't affect yours. This could be cool if used in conjunction with pagerank, so that I don't have to keep clicking on all the little "o"s...it makes it so I only have to see 1 page of links.
The biggest flaw I can see with this system is that if I'm looking for something rare and specific, once I find it, I won't thumbs-up it, I'll just click on the link...It might be useful to have a "thumbs-down all on page checkbox" which might narrow the search intelligently.
"This could be useful for those cases where Google just refuses to return the search results you want."
That has really never happened to me. Google is fast and extremely accurate, especially when you do a more advanced search, + this and - that.
I'm not sure I would want to take the time to "rate" search engine results and re-search when I can just fine-tune my search from the start.
Goodwill left the internet with the introduction of http/html.
Google's PageRank is failing miserably for commercial search. PageRank is fine for academic / informational searches.
In a commercial environment, it is simply not possible for a free search service to exist that is fair, represents an even distribution of wealth, and is immune from abuse.
Advertising has to be paid for. "Free Search" is fine for university sites and purely non-profit informational pages, but for a commercial search your position in search engines must be purchased based on the keywords against which you wish to bid.
Otherwise basic economics breaks down.
This seems like a great idea. Google might be number 1 in the search engine rankings at the moment but it would be good to see them have a bit of competition so that they do not use their dominant position for financial gain.
Here in the lab we're doing some work on using the principles of thermodynamics in order to improve search engines. The second law of thermodynamics states that in a closed system ethalpy will alway increase, which is a lot like the disorder cause by sites spamming themselves to search engines . In addition the searching patterns of users can be thought as analogous to the fermi level of a solid. In theory applying thermodynamic equations to the process of search engines should allow for more efficient algorithms to be developed. Although this has been known for some time the process involves solving some fairly hefty quadratic equations which have needed some serious computing power to process. Hopefully though a real leap forward should be no more than a few months away.
All that glitters has a high refractive index.
It is possible to delay the serving of pages that require interactive action. Then an automated robot will not be very fast.
Also, the behavioural pattern of an automated robot can be detected very easily, imagine a connection from a domain suddenly submits favorable reviews for a particular page, and no other such review is submitted. This should raise a red flag. If the effect of reviews is processed and used after an analysis, I think robots can be defeated.
ato
What is really needed is to separate out commercial sites. Google works great 90% of the time but when you are searching for something that triggers a response from sites trying to sell something, the results get swamped with the commercial noise.
This would benefit commercial sites because when you really are looking to buy something, you will be guaranteed not to be annoyed by anything non-commercial.
-- YAAC (Yet Another Anonymous Coward)
Is it that a google search for whittlebit doesn't even have a link to whittlebit.com.
If you can read this sig - the bitch fell off.
That would help, but it would have to know why they're bad to know how it would differ from other results that might be more acceptable.
Here's what I would do. First, instead of google returning the most relevant choices, it needs to be a factor of relevance and diversity. So, with the typical "apple" search, it would return some apple computer results, some fiona apple results, and some results about the fruit. All of those would be highly relevant, but it would only give, say, a few of each. You could then click on the more relevant results (if you wanted apple the fruit, you'd click on the three fruit links), at which point it would reject the others and give you more of what you want.
The key here is that it would have to give diversity in the beginning for you to be *able* to differentiate things like what you want from things you don't. This is not how google works now, I don't believe.
For what it's worth, this algorithm wouldn't be too complicated to do. I lack the programming ability, but I could do the algorithm in pseudocode (at point most decent programmers could reduce it to C++). It should be quite possible.
-Looking for a job as a materials chemist or multivariat
- Teoma
- Turbo10
- Grub (a project, not an active engine)
- WhittleBit
So, maybe we'll get somewhere after google (not that google isn't a Good Thing), after all? And.... well, Ian Clarke and his projects is/are/may soon be really rocking the world. Those include:Is supposedly more accurate than google, but I've found it to be only okay at best
"Searches the deep net" by connecting to site databases to get the most relevant info. A lot of this info, however, comes from Google itself.
A distributed search engine project. It would use tons of people's computers as crawlers like seti@home
Read the story
A giant search system for pre-existing content, aimed at corporations.
An anonomyous content-storage system that works as a giant encrypted webserver of sorts.
A search engine that learns through user interaction
A neat little AI hack that helps webmasters do their job easier
A "edge distribution network" that will optomize content distribution. It uses some Freenet Technology
Unless I read the article incorrectly, this response-feedback-accuracy was the exact cause of the problem with google as shown by msn.
Just an observation...
"The truth suffers from too much analysis"
Page rank is cool, uses distributed data to improve search results. Definately AWESOME in the search engine world.
BUT i would also like to see the distributed concept applied to searching itself. Something like this idea, but having the engine return results on what were popular click-thrus for searches. From what i can tell (IANA Google Expert) Google isn't keeping click through data on search results (they are on the adwords, but that's different). By tracking click thru data and calculating how long a user stayed at a clicked result before hitting the back button or otherwise returning to google... good insights can be learned. Aggregate this over millions of users with billions of page views... wouldn't take too long to figure out what everyone wants to see for a particular search result. Combine all of that with improving your searches by what others are searching for... i think you are talking a powerful system.
Granted this whole idea may be liable to spamming and all of that... but that's not part of the concept yet. On the surface, it seems like a good idea.
NOTE: I know other engines track click thrus, but i don't think any of them do it for non-advertising purposes.... if it's purely to improve results then cool. If it's to show you better ads, not cool.
In Yahoo and other search engines (but not Google, that I've seen), you often get a "click-through" that goes to their system before transparently redirecting to the actual URL you clicked. This is relevance feedback. It's true that the system can't determine whether you LIKED the site (aka, whether it was "relevant"), but at least it's some sort of feedback the system can use to tune.
The other most familiar type of system I can think of is Alexa (now owned by Amazon.com, and the brainchild of the Internet Archive's Brewster Kahle). With Alexa, they could count not just that you visited a site, but how long you spent and where else you went. This is at least part of the basis for Amazon's recommendation system for books and other geegaws they sell.
Can this work in a search engine? Yes, certainly. Does it mean that a search engine that implements relevance feedback will instantly be better than Google? Definitely not! There are many other things (about 20, from what I've heard) that go in to the ranking system that Google uses...Pagerank is one of them, but there are many other factors (such as term frequency, document HTML structure, etc.). Some these, notably Pagerank, work poorly on relatively small collections (in the TREC conference, people have almost never found that Pagerank, HITS or similar algorithmns improve performance with "only" a few tens of GB of Web documents -- a few million pages).
Wanna know more about information retrieval? The TREC page above is very good for state-of-the-art research reports (see the Publications area -- it's all online and free). More general texts are mostly in libraries, but one good one online is Managing Gigabytes, which covers the IR aspects thoroughly and also has lots of ideas about how to use compression in an IR system (something that I'm curious whether Google & others do).
In general, statements like that are used by people who haven't actually thought through the algorithm in detail, or who don't have good knowledge of algorithmic theory.
None taken. Put it this way - I could write it in matlab, and I could write it pretty bad in C++. However, I'm not familiar with google's code, and wouldn't be able to integrate it into that. But I could write a version of it, just not as it would need to be, final form. In other words, I'm very familiar with the algorithms involved, that's definitely not the problem. I do work on problems similar to this in grad school - the source of the data is completely different, but the same tools can be applied.
In specific, your suggestion sounds excellent. Sufficiently excellent that I would be very surprised if Google, with their famously large R&D division, didn't have some very smart people thinking about it or something similar.
Thanks, and I agree - if they're not doing this, they should be/have been. What I outlined would be reasonably accomplished through new applications of existing decision theory algorithms.
Thinking about it briefly the first couple aproaches I come up with wind up being factorial time. Plus there is a lot of fuziness as far as how to promote Fiona Apple links but not just lousy Apple Computer ones, not to mention search terms where the "families" of hits are less distinct than for Apple.
it's not as fuzzy as you'd think, and I think this could be done with less computational overhead than you'd initially believe. Basically, what we have is a classic supervised pattern classification algorithm, where the two classes are "useful" and "not useful." At the point where you tell it the groupings, then it's just a matter of determining what characteristics are common among the groups. You'd have to reduce the results to more ordinal characteristics, but this would be a solution similar to how mozilla translates emails into vectors of charactersitics for their Bayesian mail filters.
Most of this could be done starting with, say, a few hundred results or so per search. Arranging into categories from here would be fairly trivial, at which point those categories would be presented to the user. The user could then update the relationships as they are determined by the computer, and resubmit.
Of course, the more samples you use, the more overhead. Also, the more descriptors/features/parameters, the mroe overhead. Using one way of doing it, the problem would be linear with samples, and O(N^3) with features (due to a matrix inversion). Not all that bad, particularly when the number of features can be capped, and does not grow (necessarily) with samples.
-Looking for a job as a materials chemist or multivariat