Slashdot Mirror
Open Source Analog to Microsoft's Index Server?
An Anonymous Coward asks: "I have been tasked by my noble employer to find a better way accessing the 4,000 odd management documents and procedures we have. Currently MS Index Server is being used to provide a fairly good searching system. Index Server (for those that don't know) trawls through files and indexes their content.. ASP is then used to search the resulting database. My question is, there has to be a way to do this with nice open source software? Does anyone know of any competitors to index server that can index microsoft office documents? Thanks!" Might not HT://dig be a good foundation on which to build such a system?
It's not open source, but Sherlock for MacOS (part of the OS) has always featured hard drive or folder indexing features that can scan contents of documents fairly quickly and efficiently. I've not seen its performance on a /huge/ archive, though.
--Fifster
Dont google license their engine (which reads word, powerpoint etc?)
I tried mnogosearch and swish-e. Different plusses and minuses. Later on I discovered that mnogosearch has a PHP front end and can be installed from a Debian package.
My advice is to set up two entirely different search databases. Otherwise it's very difficult to compare hits, ranking performance, or discovered differences in the lexeme policy.
I realize this is a little extreme, but what the heck:
Imagine if the poster's company didn't have all their documents in a proprietary format. They would have plenty of other indexing programs available to them.
And think, if that gigantic percentage of businesses didn't have their information trapped in a proprietary information format, there'd be even MORE solutions in the marketplace to choose from.
When you don't come up with a cheaper and quicker solution, be sure to let your boss know it has just a little something to do with a proprietary format on a proprietary platform sold by a monopolist.
Happy Sunday!
Have you ever tried using an HT://dig search? I despise that search tool on the basis that the results it throws back are not ranked all that well and (this is easily fixed) ugly.
It's been a while since I've checked it out, maybe it has improved.
I had about 2GB of documentation dumped onto me for a project. The documentation had no visible structure nor any place to really start tackling it so I decided to just index it all. The documentation was on my Windows2000 machine and I put ASPSeek (a GPL'd search engine that no one seems to know about) on one of my Linux workstations. I used pdftotext and word2txt as filters and let it chew through the documentation. The results were good enough that, when I left the project and shut down the ASPSeek interface, it took about 15 minutes before someone (who already had it all indexed on his Windows2000 workstation) was at my desk trying to get me to turn it back on.
Check out this page on twiki.org: http://twiki.org/cgi-bin/view/Codev/SearchEngineVs GrepSearch -- it discusses some search engines that have been / are being considered to replace the grep based search on TWiki.
.cgi based environment quite well, and can index Word documents.
To me, Namazu and Bool sound promising, but some others are discussed there as well.
TWiki is a Perl and cgi based wiki, and Namazu seems to be able to integrate into a
Hope this helps!
Haven't tried the latter, but it may fit the bill. DocumentLibrary home
Eric Schmidt is even more of a marxist-facist than Bill G himself. Don't lose any sleep over it, though: He'll destroy Google, just like he destroyed Novell.
I highly recommend taking a look at the Apache Lucene Project, at http://jakarta.apache.org/lucene/
It's a full text search engine API, so some coding for your specific requirements would be required. However, it's fast, extremely flexible, and has a pluggable interface for documents. It comes with native support for plain text, and for proprietry document types, we've written simple wrappers around tools like "pdf2text" and "catdoc" to index PDF's and Word docs.
> Might not HT://dig be a good foundation on which to build such a system?
Hey Cliff, news flash: HT://dig suX0rz your Anal Cox!
Much talk has been made of intergrating Lucene + POI to provide indexing of MS Office Docs, but I don't what stage that is at.
Read more of this story at Slashdot.Read more of this story at Slashdot.Read more of this story at Slashdot.
I know Scot Hacker used to use Glimpse to do searches on www.betips.net. From the brief research I have done on Glimpse, it would work well if you mainly have text files.
Hello there.
I have done all of this before in a commercial environment using Glimpse and Perl.
I'd recommend you check out glimpse and webglimpse. They ought to do what you are after, for free.
Cheers
Stor
"Yeah well there's a lot of stuff that should be, but isn't"
I don't know if you'd consider using Perl, but I've had some good luck with the Fluid Dynamics Search Engine. By default it can search text and PDF documents, and after some work I was able to get it to search the text of Microsoft Word documents too.
...and anybody who's ever used it knows what I mean.
Getting the results you want is nearly impossible, and the page rendering of results is nasty as hell.
Just because htdig is OSS doesn't make it a good tool. It's old, outdated, and is one of the worst examples of OSS available today.
Try xapian, www.xapian.org, about to undergo it's first release.
It is based on an temporary open-source release of one of SmartLogik's products.
I swear by it and find it highly flexible.
I guess, though, unless you are a hacker - say capable of using to actually index your documents, you might want to wait for the next release.
I use it in preference to htdig, swish++ and others I have looked at and sadly left; xapian is very fast and easily passes the 2G limit systems such as swish++ suffer from, and supports dynamic aggregation of multiple indexes into one search!
Sam
blog.sam.liddicott.com
checkout the openfts project at sf.net
It takes advantage of the unique indexing capabilities of PostgreSQL database server.
http://www.mnogosearch.org/
is working like a charm for doing that. There is also extended functionnalities like caching (like google) and a multitude of support for external format.
Released under the GNU General Public License.
Actual humility and admission of one's flaws in a Slashdot post. That's gotta be one of them signs of the Apocalypse.