Building a Fast Wikipedia Offline Reader

ttsiod writes "An internet connection is not always at hand. I wanted to install Wikipedia on my laptop to be able to carry it along with me on business trips. After trying and rejecting the normal (MySQL-based) procedure, I quickly hacked a much better one over the weekend, using open source tools. Highlights: (1) Very fast searching. (2) Keyword (actually, title words) based searching. (3) Search produces multiple possible articles, sorted by probability (you choose amongst them). (4) LaTeX based rendering for mathematical equations. (5) Hard disk usage is minimal: space for the original .bz2 file plus the index built through Xapian. (6) Orders of magnitude faster to install (a matter of hours) compared to loading the 'dump' into MySQL — which, if you want to enable keyword searching, takes days."

Comments

Re:Why?

[rabblerabble]

I'll bite...Unfortunately, I don't have a basement, so therefore there are times that I am required to venture into the outer realm that happens to be heated by the big ball of gas known as Sol, as opposed to a pump ;P Seriously though, this is exactly what I have been looking for. What better way to show up your friends when they cry "You're wrong, google it!" knowing that there is no connection possible within twenty miles. Next time i'm drunk at the beach and someone wants to pretend to know the history of coffee harvesting, it's on.

Ho-Hum ...

[jabberwock]

What, no auto update? No User Agreement? No disabled features that are enabled by a mammoth key? No product registration?

Let us know when you're ready for prime time ... ;-)

Take that, Mr Obviously A. Troll!

[ampathee]

Programmers shouldn't be wasting time on these trivial, pointless projects. We need their work in other more important projects!

Hah! I'm going to start work on (let's see..) a random lolcat generator now, just to piss you off.

Re:Take that, Mr Obviously A. Troll!

[MarkRose]

You mean something like lolcatgenerator.com? Looks like someone already tackled that important project! lol

Just settle it the old way

[EmbeddedJanitor]

Kick sand in their face!

I hope

[Nikron]

That you don't dump the wiki at a bad time.

George W Bush

Is a dick head!!!!11

Re:I hope

[Anonymous Coward]

You mean before someone makes it inaccurate again?

Oh, nevermind, I see the problem:

George W Bush

Is a dick head!!!!11

should be

George W Bush

Is a dick head!!!!!!

Man, those out to mess with the content are getting more and more subtle...

Hitchhiker's guide here we come!

[Brietech]

Combine this and one of the new E-ink ebook readers, make it pretty rugged, slap a solar panel on the back and man. . . you have something really close to a genuine hitchhiker's guide to the galaxy. Ah, I love where technology is heading =)

Re:Hitchhiker's guide here we come!

[Sneftel]

As long as hitchhikers primarily need to know how to evolve a Pikachu into a Raichu, and how Benjamin Disraeli has been referenced in pop culture.

Re:Hitchhiker's guide here we come!

[RandomWhiteMan]

You laugh now, but just wait until you're stranded in the middle of Blackheath England, needing a ride from a conservative British History Scholar who has his son with him playing Pokemon Gold. Won't be so smug then, will you. I bet you won't even have your towel on you when this all goes down.

Re:Hitchhiker's guide here we come!

[Gromius]

Yes its a perfect fit. Particularly as Wikipedia has now supplanted the Encyclopedia Britannica in many places as the standard repository of all knowledge and wisdom. Although it has many omissions, contains much that is apocryphal, or at least widely inaccurate, it scores over the older more pedestrian work in two important ways.

        * 1. It is slightly cheaper
        * 2. It has the words "You can copy and edit me for free" inscribed in large friendly letters in the license.

Also like the guide, although it cannot hope to be useful or informative on all matters, it does make the reassuring claim that where it is inaccurate, it is at least definitively inaccurate :)

Re:Hitchhiker's guide here we come!

[nstlgc]

Just so we're clear, you can make Pikachu evolve into Raichu by using the Thunder Stone (which makes sense, since they're Electric Pokémon). However, due to the emotional value Pikachu has to trainers, most of them choose not to evolve him. Some Pokémon games even plain don't allow this. I hope this was helpful.

Re:Uh....

[Gazzonyx]

I love programming useless things just for the challenge.

Have you ever worked on a project called "Clippey", by chance?
No, he said he has a love for programming; not a seething hatred for users. Besides, everyone knows programmers only hate admins. ;) On behalf of the programmers, I'd like to say that this isn't true we love our admins. Who else makes sure that our connections*&#^$: Connection Reset By Peer

What??

[icydog]

TFA is:

1. Not a thinly-veiled attempt to advertise a crappy product
2. Not bashing Microsoft
3. Not about somebody who is trolling open-source (i.e. SCO)
4. Not about Bush taking away all our rights and ending freedom
5. Not about voting fraud and the end of democracy/America/the world
6. Not decrying Vista DRM and its ties to the MAFIAA
7. Posted on Slashdot

Furthermore, TFA is interesting and informative.

Am I in heaven?

Re:What??

[Pollardito]

it'll get posted again tomorrow just to maintain expectations