Wikia Acquires Grub, Releases it Under Open Source

An anonymous reader writes "During a keynote address at the O'Reilly Open Source Conference (OSCON), Jimmy Wales announced that Wikia has acquired Grub, the original visionary distributed search project, from LookSmart and released it under an open source license for the first time in four years. Grub operates under a model of users donating their personal computing resources towards a common goal, and is available for download and testing."

Comments

FIST SPORT

[ringbarer]

Oh fuck! Does this mean I have to read [Citation Needed] every time I boot up?

Re:FIST SPORT

[Archangel Michael]

Not the bootloader

Re:FIST SPORT

[Ltar]

How do you expect him to get first post if he reads the summary?

Lunch Time

[n1ckml007]

That's funny, I ALSO just acquired some grub, yum yum.

wait a second

[wwmedia]

WIKIA is a for profit company!

so they want to use other peoples spare CPU cycles to build an empire on top without spending money on servers?

rofl!

all them spare cpu cycles would be better used for distributed research like Folding@Home and other @home projects

Re:wait a second

[kebes]

I use Google search, which is produced by Google Inc., a for-profit company. I use the search "for free," but then again I "pay" with my eyeballs, since the search pages have ads. This is a trade-off that I (and many others) are willing to make.

So why is it so crazy to think that users would be willing to participate in a search engine where you "pay" with your spare CPU cycles? If the search engine generates useful results, it seems like a fair trade-off for me.

I'm not sure what Wikia's business model is here. It's probably not ads, since it would be difficult to reliably enforce and bill an ad-pushing system using software that is open-source and a network that is peer-to-peer. Probably they hope that this will drive more traffic to Wikia projects, or somesuch.

But, ultimately, I don't see what's so crazy about a for-profit company and end users coming to a mutually beneficial agreement. I donate CPU cycles and bandwidth, and get access to search results. Sounds fair to me. If the result is useful and the terms-of-use not onerous, most users will happily use it.

Interesting

[Waffle Iron]

I just loaded this new grub on my system. I got a strange new boot menu:

1. Fedora Core 7 (redirected from Red Hat)
   [Note: This selection may be too technical for the average user. Please help revise to improve.]
2. Microsoft Windows Vista (redirected from Longhorn)
   [Note: This entry has been locked from new and anonymous users due to ongoing controversy. See also: WGA]
3. Microsoft Windows 98 Second Edition (redirected from Win98)
   [Note: This OS requires cleanup to conform to quality standards. Please get involved.]
4. FreeBSD 6.2
   [Note: Link appears dead. This has been tagged since July, 2007]
5. MSDOS 5.11(redirected from DOS)
   [This operating system is a stub. Please help to expand and improve it. This entry has been tagged since 1981]

Use the up and down arrow keys to make a selection, press Enter to boot.

Is this a good thing?

[Animats]

Wikia is a for-profit company. Users running portions of their crawler should be paid. At least in stock of the company. Otherwise it's a ripoff. It's reminiscent of Kazaa's approach to "peer to peer": user machines do the work; Kazaa collects the money.

Distributing the web crawl isn't that big a win. The crawl is a batch job, but replying to search requests is a near real time application. The expensive part of a search engine is the system that generates fast search responses. That's where you need the systems with gigabytes of RAM and tight coupling to the other machines of the cluster.

Doing the web crawl on user machines offloads some of the effort, but not all that much of it. If you want to cut crawl costs, some of the query machines can be devoted to crawling during slow periods.

Remember, you can't trust the client. Web spammers can modify their copies of the crawler to report extra, phony links to their web sites and boost their stats. This gives a whole new meaning to the term "link farming". Until Wikia, there was no easy way for "search engine optimization" types to mess with the internals of the search engine. Now there is.

Besides, what's the selling point? "Our search costs less to use than Google?" Hello?

Release

[EnsilZah]

Yes, but in what form are you planning to release it?

Re:Release

[Anonymous Coward]

Binary blob.

Re:Release

[Mathness]

It will be released as public domain, delivery will be as a Fast Universal Dump and Gas Exchange (fudge) packet [1] over a Wireless Connection (WC) with a water protocol to a local STP node (sewage treatment plant). At the STP node it will be processed for redistribution.

[1] including several pamphlet with Rorschach-like art.

Welcome Back Grubby

[zokord]

I'm the guy that started Grub back in 1999. In 2003, after getting a little bit of press, I sold the company to LookSmart. I was hoping for a continuation of the OS license for Grub, and the financial backing of a larger company that could help develop the product out to it's logical conclusion - distributed, open search.

Unfortunately that didn't happen with the situation, and I decided to move on to other opportunities. Now here I am again, and I fully support what Wikia is doing with Grub, and what their resources can do for the project and the problem it can solve.

Myself (Kord Campbell), Igor Stojanovski and Ledio Ago (both who work at Splunk BTW) are three original founders of Grub. We are now helping Wikia out with getting it up and running, and explaining how things work (or don't) and will continue spending a bit of time helping out where we can as the project matures.

I would like to point out that Grub itself isn't all that interesting right now. About all it does is distribute jobs that consist of URLs to crawl. Yes, something similar could be done with BOINK. Yes, nothing is being done with the crawled data. Yes, it breaks occasionally and it's full of bugs.

However, it's a start. It's the first pass at fully distributing the job of search, and putting it where it belongs - in the commons. Search doesn't belong to Google, or Wikia, it belongs to everyone. It's your data, and it should be your search engine crawling, indexing and searching that data - not some monolithic profit hungry company.

Go and read the page on search over at Wikia: http://search.wikia.com/ - Jer Miller (worked on Jabber) explains what they have in mind for Atlas. It's a fully distributed, OS, open protocol dream of making better search. Like Wikipedia (which is non-profit), Jimmy Wales wants search to be open, and community driven/managed - it's not about making gobs of money off your CPU/Bandwidth - it's about making better search for everyone.

Ideally the current Grub clients/server will go away, and be replaced with something better. For now, you have to crawl before you walk, and you have to walk before you run. Given time, and support from the OS community, I'm sure Wikia will do the right thing here.

If you want to get involved and help out, start by hitting the wiki and contributing your thoughts. We are going to need coders working on different aspects of the project as well, so think about volunteering in your particular area of expertise.