Slashdot Mirror
New Search Engines
An anonymous reader wrote in to say "It seems that there's a new company out there
called Fast Search
and Transfer which is competing with Inktomi.
They have a demo online at www.alltheweb.com and their engine seems to be ultra-fast.
News about this is available here.
Try out the demo, it is awesome what these guys have done."
It is fast, but so far I've not had as good luck searching as
with other engines. And the speed is probably largely due to
the sparse HTML. But its not bad.
As soon as they get much closer to their "all the web" idea, I'll take another look. But for now my default engines will stay.
Anonymous Cowards: Proving daily that human beings are innately jerks.
Posted by FascDot Killed My Previous Use:
I clicked the button and I had results before the mouse had fully un-clicked.
That's damn fast!
There's gotta be a reason apart from a fast machine. I have this feeling that the database ain't so big, making the searches incredibly speedy.
Still, it's a great speedy thing to see.
Try searching for "shareware" and check out the
results -- seems they don't discourage keyword
spammers!
It looks to be by the same guys who created Fast FTP Search... which went downhill when it was bought by Lycos.
As someone else commented, these people wrote the search engine behind ftpsearch.lycos.com. Which is fast. There are a few more reasons, apart from sparse HTML.
This engine asks for no cookies
The output is not in a table, so you see the results as they arrive in your browser, without having to wait for the whole table (in lynx it makes no difference
The load is not very high yet, probably.
But having seen FTPsearch in action for the last 4-5 years, and having seen it always return results quickly, it wouldn't surprise me if alltheweb stayed fast.
"There are two major products that come out of Berkeley: LSD and UNIX. We don't believe this to be a coincidence."
This technology isn't very new.
Parts of it at least were developed at the Norwegian Institute of Technology, and used in the search engines FTPSEARCH and MP3Search.
But it's really nice to see them get the publicity they deserve.
Harald
The only search engine I know that does a good job at this is Google. It is so good at finding relevant sites, I don't care if the response time is occasionally a little slow.
(Google uses a nice algorithm where they gauge the "importance" of a page by how many other sites link to it.)
If a thing is not diminished by being shared, it is not rightly owned if it is only owned & not shared. S. Augustine
Damn this engine is fast! And I love how the searching syntax is spelled out on the first page. This is really phenomenal. Certainly my new default search engine.
that alltheweb.com is *fast*. good coverage too.
--
Wage Slave Journal
Egosurfing, I found it quite refreshing to NOT see my old web pages that died 4 months ago, yet found a large number of hits to people I'm apparantly related to. (gasp) Still no sign of my current web page tho. Curious. More time perhaps.
But did anyone else find there's number of identical links? (I'm used to metacrawler, so maybe this is normal for search engines?)
Friends help you move. Real friends help you move bodies.
I wonder if these means that Dell is going to start supporting FreeBSD on their machines as well.
-Jae
Interesting to notice, their HTTP headers proclaim the site is running FreeBSD. It is nice to see FreeBSD and Linux being used in large commercial applications.
I find this really hard to believe but CNET is doing spamdexing.
yeah, off the bat i recognized the logo from the old ftpsearch.ntnu.no, now ftpsearch.lycos.com. are they a part of lycos, is lycos partnered with them, or what?
The specs on the search engine are available at http://web.fast.no/product/search/d et.asp?id=34.
The press release doesn't exactly scream it out, but the search engine is actually just a little bit of software stuck on top of some pretty neat custom hardware. They call their chip the FAST PMC (Pattern Matching Chip), and their server is just your average (well, sort of average) high end server, with a buttload of those chips stuck on PCI cards.
The specs on the PMCs are available at http://web.fast.no/product/PMC/det.asp ?id=52.
FAST claims 100 MB/sec throughput on each chip, and each card has its own RAM (from 8 MB to 2 GB). The chips actually run at 100 mHz each, and even have support for RegEx matching (slightly limited).
From the specs:
A typical configuration will contain 4 to 8 plug-in cards per search node, and 16 or 32 chips on each card.
Overall, I'm pretty impressed - putting search capabilities into hardware is a pretty good idea, especially since so much of a modern processor is geared toward things like Floating Point calculations, which doesn't help text searching at all.
Scott Severtson
Software Developer
Auragen Communications
scotty@auragen.com
Scott Severtson
Senior Architect, Digital Measures
If a search takes exactly 0.0050 seconds, it's because it's cached. The engine caches all searches for some time, and if someone does the same search again, the results are simply fetched from the cache.
"...have constructed an advanced search capability using a high-performance, low-cost software/hardware combination...."
:)
That little quote made me wonder what they were running. Why didn't they just say? They didn't seem at all shy about mentioning the Dell 4300s.
If the web servers are running FreeBSD, I wonder if it's also powering the database servers.
Maybe I'm reading too much into their vagueness here. It seems to me that companies which have close ties to MS (like Dell) are a little reluctant to trumpet the virtues of other operating systems too loudly. As a result, I tend to think:
#ifdef PRESS_RELEASE
Save the whales. Feed the hungry. Free the mallocs.
I found unique info to plagarize for the paper I'm writing.
FAST is a norvegian company that specialize in fast algorithms for search and also transfer of video and images. Since FAST is a research company they are interested in selling their technology, not productifying it themselves. All the web is a marketing site for FAST technology and is probably going to be closed down after an eventual sale (according to norvegian newspapers).
The company has a couple of strange fellows working for them, there is former archeologist, medical doctors and statisticans. Guess a combination of their speciality fields spurred a really strange, but fast, algorithm.
Hope that will answer som questions.
By the way, the URL to fast is www.fast.no.
I just did a search for my real name and came up with a ton of accurate links, more than any other engine in the past. Hot damn, ain't that fancy? =)
What can I say? I've bookmarked this engine.
It is still the dark of night.
Nice, fast powerful, whatever... yet another Altavista with maybe one day more relevant results.
Still, for us, common mortal beings and small company sites (50'000 to half a million pages), maybe a decent accessible search engine like Alkaline is more than enough! It's free!
cheers
dB.
dB@dblock.org
That's how Alkaline works. But for now an index of 179734 distinct word forms and 21309 pages eats ~40 MB together with the engine and serveral fast search structures. It grows slowly, so for 40'000 pages it will not get over 50-60 MB.
:)
Anyhow, that means your next purchase of the first terabyte of RAM will make the shop happy
cheers
dB.
dB@dblock.org