Slashdot Mirror

I've found that Lucene is useful for creating custom search/indexing solutions, but as a server, I much prefer Sphinx. It's lighter-weight and compatible with pretty much any language. It's also remarkably fast.

Sphinx Search works quite well, is very fast and can handle very large datasets.. Only down side is the indexes are not live.. http://www.sphinxsearch.com/

...Sphinx for lightning-fast searches (and stemming, and relevancy, and much more) and Open Calais for text analysis. Combine this data set with those two tools and you could have a pretty nifty site.

I've plugged this before... but Sphinx is a great full text search engine. I've helped with a couple of production deployments and folks have been happy with it. The Ruby on Rails integration is good and the API is easy to use... for a simple demo including excerpt highlighting, try some searches on my military reading list site.

I've plugged this before... but Sphinx is a great full text search engine. I've helped with a couple of production deployments and folks have been happy with it. The Ruby on Rails integration is good and the API is easy to use... for a simple demo including excerpt highlighting, try some searches on my military reading list site.

...I've been using Sphinx a lot recently and have been really pleased with it. The indexer is fast, there's good Ruby on Rails integration, and I don't worry about scalability since if it's good enough for craigslist it's good enough for me. Definitely worth a look for your next project that needs to do full text search.

For a quick demo of it, do some searches here.

...I've been using Sphinx a lot recently and have been really pleased with it. The indexer is fast, there's good Ruby on Rails integration, and I don't worry about scalability since if it's good enough for craigslist it's good enough for me. Definitely worth a look for your next project that needs to do full text search.

For a quick demo of it, do some searches here.

> One of the biggest aces in the hole was PostgreSQL.
> The cost for us to come in, set up and install everything
> was cheaper than some other well known DB vendor's cost of database software alone.

Right on. Same goes for Sphinx vs any proprietary text indexing setup I've used. And it's fast, and there's good Rails/Ruby support via UltraSphinx and Riddle.

Whatever you've got, consider replacing it with Sphinx, which is awesome. I'm using it with Rails and the Ultrasphinx plugin and it's been great - doing excerpts (for example, notice the highlighted results from a search for 'combat') - was a piece of cake.

FYI, there is a way to search InnoDB tables - Sphinx Search, and it's pretty fast too.

I'm not suggesting this as a solution to your problem since there are a fair number of off the shelf solutions, but for reference you if you were going to create something using an API such as Lucene, check out the awesome open source Sphinx: http://sphinxsearch.com/

It's very powerful and very fast, but you'd have you figure out how to convert the pdfs, the docs, etc into a format that it will take. I'm currently using it for a scientific journal article system. I'm not indexing full text of the articles but I am indexing the citation information including abstracts. It works very well for my purposes.

If you want to improve the performance of your database, you should check out Sphinx to handle your fulltext searching. In my opinion this is one of the most criminally overlooked pieces of software when it comes to website optimization.

Don't use MySQL's fulltext indexes. Instead, use Sphinx (http://www.sphinxsearch.com/) which is incredibly fast and indexes your MySQL database directly. You can even query it through MySQL using a custom storage engine.