Slashdot Mirror

← Back to Stories (view on slashdot.org)

Building a Bigger Search Engine

Posted by ryuzaki0 on from the size-isn't-everything dept.

skreuzer writes "Wired is running a story about a distributed web crawler called Grub. People who choose to download and run the client will assist in building the Web's largest, most accurate database of URLs. This database will be used to improve existing search engines' results by increasing the frequency at which sites are crawled and indexed. Conceivably, Grub's distributed network could enable state information to be gathered on every document on the Internet, each and every day."

2 of 278 comments (clear)

  1. Search engine software and lack of A . I . by zymano · · Score: 0, Troll
    Have these search companies ever thought of actually hiring people to sort some of the garbage in their databases?

    of course not.

    These guys buy a bunch of servers and let some DUMB designed software to try and find what your looking for. It's really stupid shit if you ask me.

    Whats really needed is some sort of A.I. . Either real people actually giving you advice on finding something or programs that can think like humans and know what humans want.

    Also needed is to catagorize FREE CONTENT from the commercial websites . The internet which was invented for government sharing information has turned into a FILTHY ,CRASS, Commercial overloaded sack of SHIT.

    Anyone that can implement these features , I tip my hat to you. Google, yahoo, lycos all suck in my opinion. These companies should hire some of those soon to be out of work telemarketers but they wont because they think their software is so special but it's actually rediculously unprofessional,shoddy and cheap.

    If anyone wants evidence then try and find content in these search engines about starting an internet business. I get SPAM site after spam. Nothing legitimate. It doesn't help that these overlyhyped so-called searchengines take under the table cash from businesses for placement on their searches.

  2. Re:Will Grub take off or be smashed? by Kragg · · Score: 0, Troll

    the client is open-source

    Really? My god, how exploitable is that. Give me a week and any searches for sex, news, books, shopping, flatulence, football or art will all lead directly to my ad-filled spamsite.
    Where's my IDE..?

    --
    If you can't see this, click here to enable sigs.