Slashdot Mirror

← Back to Domains

Domain: webglimpse.net

Stories and comments across the archive that link to webglimpse.net.

Comments · 9

  1. Re:Linux anyone? by rusty0101 · · Score: 1 · on Desktop Search Engines Compared

    I tossed a couple of notes down thread, but two options for Linux users include ht://dig http://www.thdig.org/ which presents exclusively a web based interface, and glimpse/webglimpse (see http://webglimpse.net/ which provides both a commandline mode search as well as a web based search.

    Enjoy,

    -Rusty

  2. Re:Searching file content! by edesio · · Score: 1 · on Yahoo! Releases Desktop Search Tool

    One should try glimpse by Udi Manber actually in yahoo!

    It can index and search for words, substrings, and sentence. It handles multi-line records, do approximate searches, and is pretty smart and fast.

  3. Re:standard filesystems are NOT databases by kirkjobsluder · · Score: 1 · on Database File System
    therefore, doing searches on a relational database filesystem (find me all music files with dates between last week and last month: SELECT * from files WHERE files.type = "music" and files.date NOW() - 7days

    you _can't_ do that sort of thing on a traditional filesystem.

    Ahem:

    find . -name *.oog -mtime -7

    That's just for the bad SQL you posted (I imagine there must be a missing operator in the where clause between "files.date" and "NOW()".)

    A better example of where indexing would be would be useful is a case where you want to index the content of files. For example "I want a list of all papers authored by me where I've referenced Doe, Doe, and Doe's seminal paper 'Mental Masturbation About the Need for Relational Database Filesystems on Slashdot'". But even then, this is a type of indexing that can be easily accomodated within a heirarchal database. (For two examples, glimpse and swish-e come immediately to mind.)

    To answer your two questions:

    some people mentioned here that they already organise their files. great. fantastic.

    HOW LONG DID IT TAKE YOU?

    A pretty trivial amount of time. Each new project starts with:
    mkdir newproject
    cd newproject
    A maximum of 1 minute, even with my gimpy hands. This is about the same amount of time, and perhaps much less, that I would have to spend adding the keyword "newproject" to every file that gets added to the project.

    and how long would it take to reorganise?

    Well, it depends on the reorganization. But for example, I reorganized one of my key project directories (version controlled in subversion) with.
    svn mv proposal-may2004 proposal1
    svn mv proposal-aug2004 proposal2
    svn cp proposal2 proposal
    svn commit
    (Making two archive "tags" and starting a new branch for the "HEAD" tag.)

    A nice thing about heirarchal file systems is that you can reorganize on multiple levels including collections of files. ("proposal" in my case includes about a half-dozen files.) I would argue that it would take me just as much time to recode those files using database keywords as renaming the directories they reside in.

    A large part of the argument for relational database filesystems seems to be that the same types of people who are unwilling to do the work necessary to create good heirarchal file trees, will be willing to do the work to attach the metadata needed to replace heirarchal file trees. Switching from a heirachal to relational model is not going to change the GIGO rule, and the solutions around the GIGO problem (such as full-text searching, and journaling) don't depend on either model.

    A heirachal model also provides some nice facilities for dealing with related collections of files. It is unclear no me how they would be implemented in a relational model, or how a relational model would deal with stardard filenames like "README", "CHANGELOG" and "index.html".
  4. Glimpse by polymath69 · · Score: 1 · on A Grep-like Utility That Works on More than Text?
    No one has mentioned glimpse, probably because they moved to a non-free license a while back, but the 3.0 version is still gratis. And it allows you to configure filters to search files of whatever type you like.

    Here's a snapshot of the source for Glimpse 3.0, packaged up from my system as I don't have the original tarfile anymore.

  5. Glimpse by polymath69 · · Score: 1 · on A Grep-like Utility That Works on More than Text?
    No one has mentioned glimpse, probably because they moved to a non-free license a while back, but the 3.0 version is still gratis. And it allows you to configure filters to search files of whatever type you like.

    Here's a snapshot of the source for Glimpse 3.0, packaged up from my system as I don't have the original tarfile anymore.

  6. Re:Reply to my own question: non-HTTP indexing by Jason+Pollock · · Score: 1 · on Indexing and Searching Text and PDF Repositories?

    If ht://Dig does most of what you want, why not make it understand file:// URL's if it doesn't now? I wouldn't expect it to be _that_ difficult an addition, especially if you are concerned about the availability of source code.

    Otherwise, why not just pay up the US$2k and buy a Glimpse site license? US$2k is probably worth about _two_ days of your time to the company (depending on the size of the company). That would cover everything as long as you kept it to one site.

    Glimpse also appears to be available with source too, check out the bottom of the download page.

    If the tool is to be used to create a commercial product that you are going to sell, then you won't be covered by that. Personally, if I was going to be using a GPL'ed (or other open source) tool as the core of a product I was going to sell, I would feel ethically bound to pay some money for it, regardless of it's availability for "free".

    Jason Pollock
  7. Glimpse / Webglimpse by polymath69 · · Score: 1 · on Internet Searching Using Regular Expressions?
    Some sites are indexed for quick searching using Webglimpse, which supports modified RE searching. There's a short list of such sites here.

    Unfortunately I don't know of any Web-wide RE-capable database. Here's hoping someone downthread does...

    --

  8. Re:Performance? by dougmc · · Score: 1 · on Internet Searching Using Regular Expressions?
    However, doing the prelimiary match using the regexp would definately be resource-prohibitive. In the above example, you would have to read the text of each file in to do the regexp. Not to mention the cost of keeping the text around.
    You do it by looking for the words in the regex, then you search for those (the words can be indexed -- the regex can't), then you apply the regex to the results to further narrow them down. This his how glimpse/webglimpse (www.webglimpse.net) works.

    However, this all falls apart when the keywords are all very common, because nearly every page everywhere will contain them, and so the actual regular expression search will have to search thousands of pages. But for uncommon words, it works fairly well.

    glimpse does do this, but it has problems with memory usage and being slow at times -- probably exactly because it does this.

    Sorry, but I don't know of any full-Internet search engine that allows this. Your best bet is probably to write something that looks for the keywords in a regex, feeds them to google, then downloads every page that matches and then runs the regex on your own computer to further narrow down the results. Depending on how common your keywords are, it may work well, or it may try to download half the Internet.

  9. Free, scalable search engines by Anonymous Coward · · Score: 1 · on Juggernaut GPLd Search Engine
    Having just finished a research project on the state of the marketplace in search engines and related technology, I'll be interested to see how Juggernaut stands up in practice. For anyone interested in the best of what's already out there for free, at least under some conditions, I suggest checking out the following:

    - the IB project at Basis Systeme netzwerk,

    - the former commercial products from PLS that AOL is now giving away,

    - the ASF project,

    - the Webglimpse pages,

    - the pages for the mg system.

    For a comprehensive presentation on the subject, see the searchtools site.