New Search Engines

An anonymous reader wrote in to say "It seems that there's a new company out there called Fast Search and Transfer which is competing with Inktomi. They have a demo online at www.alltheweb.com and their engine seems to be ultra-fast. News about this is available here. Try out the demo, it is awesome what these guys have done." It is fast, but so far I've not had as good luck searching as with other engines. And the speed is probably largely due to the sparse HTML. But its not bad.

Comments

Hardware-Based Searching

[scottsevertson]

The specs on the search engine are available at http://web.fast.no/product/search/d et.asp?id=34.

The press release doesn't exactly scream it out, but the search engine is actually just a little bit of software stuck on top of some pretty neat custom hardware. They call their chip the FAST PMC (Pattern Matching Chip), and their server is just your average (well, sort of average) high end server, with a buttload of those chips stuck on PCI cards.

The specs on the PMCs are available at http://web.fast.no/product/PMC/det.asp ?id=52.

FAST claims 100 MB/sec throughput on each chip, and each card has its own RAM (from 8 MB to 2 GB). The chips actually run at 100 mHz each, and even have support for RegEx matching (slightly limited).

From the specs:
A typical configuration will contain 4 to 8 plug-in cards per search node, and 16 or 32 chips on each card.


Overall, I'm pretty impressed - putting search capabilities into hardware is a pretty good idea, especially since so much of a modern processor is geared toward things like Floating Point calculations, which doesn't help text searching at all.


Scott Severtson
Software Developer
Auragen Communications
scotty@auragen.com