Slashdot Mirror

Also of potential interest are a couple of links at the bottom of each search results page [to] try your search on AskJeeves.com or DirectHit.com. [I]t seems somewhat odd that they'd include links to what most people [...] consider to be inferior search engines instead.

Complete the thought. Ask Jeeves, Inc. owns both Teoma (September 2001) and Direct Hit (January 2000). The selected URLs prominently display that owership relation.

From the Teoma search page:

"Teoma delivers three types of search results Web Pages: Authoritative sites relevant to your search term. Web Pages by Topic: Top result pages are grouped based on their topics. Experts' Links: Pages contain directories of links for related general subjects."

Okay, great... but where's the "advanced search" option (such as Google's, at this page)? I know this is a "beta version" of the Teoma site; maybe their advanced search functionality isn't ready for prime time just yet. Or, maybe I've got it all wrong... do they believe their engine is good enough to eliminate the need for advanced search functionality?

Also of potential interest are a couple of links at the bottom of each search results page. These links let you try your search on AskJeeves.com or DirectHit.com. As I understand it, they're gunning for Google as their biggest competition, but it seems somewhat odd that they'd include links to what most people (at least people I know) consider to be inferior search engines instead.

Just a couple of thoughts :).

How was the above comment rated troll? The word USian is not found at dictionary.com. I did not make any false statements, so it is not a Troll. I did not make any personal attacks nor is it likely to draw flames. Yes it is a bit of a slam, but then everyone should uphold proper English.

Also, DirectHit is what's powering the beta version of HotBot - which incidently uses the Cyc Engine & KnowledgeBase.

The Cyc project seems very cool - but I've not even seen any of their stuff running yet. (Despite attempting to implement an inference engine myself around the core of their KB!)

Whilst we're on that subject, the core of their KB is available for free download & usage - even in commercial apps - providing you give them the necessary credit in the app (etc). Check out their website for full detalia...

HTH,
fRoGG

This is how Direct Hit works...

How would the server verify this? Rather than presenting links to the actual sites, would it present a list of CGI's that a browser could then click, causing the server to verify the page prior to passing it back to the client? That would be a major CPU killer.

Do a search for something on DirectHit and you'll see the little people icons. The more people icons, the more popular the page is.

The extension to this is that if you find one link that's bad, they should check all the other URLs for that site. There are a ton of bad links to old pages at jargon.org, for instance.

The main problem with search engines, is the results are usually polluted with porn, warez and other such stuff, by people with loads of METAs or whatever the search engine looks for. What we need is a search engine that starts (from either a directory, or a classic search engine), and re-ranks the results according to how useful people found the site they clicked on.

But, wait a minute, they're here! There are several sites that use a popularity-based reranking system, excite and snap.com being among the most popular. Of course, the other engines are following in their footsteps - it appears that direct hit (who power hotbot and several others are using the same sort of thing.

It'd be even better if it could group users into profiles according to their (user-selected) demographic (eg: doctors, british). That way, if an American types "football" they get links about American Football (gridiron?), if a Brit types "football" they get links about soccer, and if an Australisian types it they get links about rugby.

Search Engine Watch has an article about such a system, by the same people who provide the technology behind snap.com (disclaimer: okay, I work at globalbrain, but I'm talking generally).

The main problem with search engines, is the results are usually polluted with porn, warez and other such stuff, by people with loads of METAs or whatever the search engine looks for. What we need is a search engine that starts (from either a directory, or a classic search engine), and re-ranks the results according to how useful people found the site they clicked on.

But, wait a minute, they're here! There are several sites that use a popularity-based reranking system, excite and snap.com being among the most popular. Of course, the other engines are following in their footsteps - it appears that direct hit (who power hotbot and several others are using the same sort of thing.

It'd be even better if it could group users into profiles according to their (user-selected) demographic (eg: doctors, british). That way, if an American types "football" they get links about American Football (gridiron?), if a Brit types "football" they get links about soccer, and if an Australisian types it they get links about rugby.

Search Engine Watch has an article about such a system, by the same people who provide the technology behind snap.com (disclaimer: okay, I work at globalbrain, but I'm talking generally).

The main problem with search engines, is the results are usually polluted with porn, warez and other such stuff, by people with loads of METAs or whatever the search engine looks for. What we need is a search engine that starts (from either a directory, or a classic search engine), and re-ranks the results according to how useful people found the site they clicked on.

But, wait a minute, they're here! There are several sites that use a popularity-based reranking system, excite and snap.com being among the most popular. Of course, the other engines are following in their footsteps - it appears that direct hit (who power hotbot and several others are using the same sort of thing.

It'd be even better if it could group users into profiles according to their (user-selected) demographic (eg: doctors, british). That way, if an American types "football" they get links about American Football (gridiron?), if a Brit types "football" they get links about soccer, and if an Australisian types it they get links about rugby.

Search Engine Watch has an article about such a system, by the same people who provide the technology behind snap.com (disclaimer: okay, I work at globalbrain, but I'm talking generally).

I am an engineer with a Dutch search engine, and I had this discussion with my colleagues once: whether or not we should add link-redirection. We had a very good technical reason to do so, since the poularity of links can be used to improve the relevance ranking of the engine big time (cf. DirectHit). Now, this is a much better reason than just "adding another layer of abstraction". However, the proposal was immediately discarded by my superiors, on privacy grounds. They felt people were much too itchy about this stuff and we'd better leave it alone.

I am positive that Dejanews knew exactly what they were doing, and what the risks involved were.

It is not coincedence that a problem like this pops up at a company like Dejanews. Dejanews' core business has always been on the verge of privacy violation. We all love Dejanews because it helps us tame the mind boggling amount of information that flows through usenet every day. And DejaNews' value will only continue to increase as the years go by. Imagine what a valuable research tool it will be to the future anthropologist trying to trace the evolution of certain memes through the history of internet.

However, there is a darker side. The same power that we have all come to love allows us to trace individuals just as easy as those interesting memes. And you don't need a subpoena to do so. Imagine the amount of information you can find about yourself on DejaNews in fifty years! Even if you are a mildly active usenet personality, your whole life will be out there, ready to get datamined by any dirt-digger, biographer, stalker or power-hungry megacorporation.

Sure, it's possible to "trick" DejaNews by using different aliases or email addresses. But that is a major pain in the *ss (try teaching your mum just how to do that), and forces you to actively defend your privacy instead of being able to trust yourself to remain reasonably anonymous. (And besides that, you can pretty sure that within a couple of years there will be plenty computing power to recognize a poster just by her verbal fingerprint instead of her email address. Think spelling errors here: how many people know how to spell "potatoe"?)

Dejanews has been a mixed blessing right from the start. It feeds on semi-private information and offers us a great tool in return. What we witness with the mail-click thing, is that people are irritated at the fact that they don't get anything in return for this information, not at the bare fact that their privacy is violated. Their privacy has been systematically violated by Dejanews all along, and they didn't really care.

Maybe we should have something like robots.txt for usenet. That would help, at least a little bit.
--