Slashdot Mirror
Squeezing a Wikipedia Snapshot Onto an 8GB iPhone
blackbearnh writes with this excerpt from O'Reilly Radar "Think about Wikipedia, what some consider the most complete general survey of human knowledge we have at the moment. Now imagine squeezing it down to fit comfortably on an 8GB iPhone. Sound daunting? Well, that's just what Patrick Collison's Encyclopedia iPhone application does. App Store purchasers of Collison's open source application can browse and search the full text of Wikipedia when stuck in a plane, or trapped in the middle of nowhere (or, as defined by AT&T coverage...)"
There. Fixed that for you.
Well, I could if I had an iPhone, sounds like an impressive achievement though, but how much space do you have left over after it?
Laughter is the best medicine, except if you have a broken rib.
Will somebody please build a emulator for iPhone apps!
It's a shame to have all this development go wasted on a rather expensive and fragile device with a huge monthly cost.
Thank you!
You can't live without the images!
This is easily doable.
Once you trim the earth reference down to "Mostly harmless".
Corporation, n. An ingenious device for obtaining individual profit without individual responsibility. - Ambrose Bierce
1. Goes to foreign country - one that he has never visited before
2. Doesn't have wireless access.
3. Instead of wandering about the country he spends most of his time programming ("Then basically, I spent a significant fraction of my time there in Japan, again, in 2007 writing those applications") an application so he can look up stuff about the country he isn't spending much time actually visiting.
I bow before you sir. Awesome.
Faster! Faster! Faster would be better!
xkcd comic reference
Yeah, pretty much you're turning your iphone into a hitch hiker's guide to earth, or at least america and europe if you can manage to squeeze wiki-travel onto it.
Is it sad that I am more likely to recognize you and your posts by your sig than your name or UID?
Seen that, done that been Got the t-shirt in 1978
http://xkcd.com/548/ That day is here.
Real men read Slashdot articles at -1, bottom up.
Are you crazy? Imagine the frustration when you find this horrible typo and you can't fix it!
You can fit as much as you damn please if you leave out all the media.
This is nothing new. Wikipedia has been available for several years now in MDict format: http://www.octopus-studio.com/product.en.htm
What about my palm/mobil phone/netbook/olpc/etc/etc?
And for those preferring accuracy and editorial responsibility :
http://www.ipodnn.com/articles/08/02/27/britannica.on.iphone/
How are the snapshots taken? What if at the time of taking a snapshot there is a vandalized version of an article?
... so clearly this app will never make it through Apple's review process.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.
It's cool but not $10 cool. I use 2 free apps that let me access wikipedia. Nothing really new or radical about this app unless wikipedia is really much larger and the author managed to cull 2gb from it.
It would be nice if he shared/donated some of the profits from this to Wikipedia, seeing as he's getting the database for free. There didn't seem to be a mention of it in the article or his personal site.
I dreamed of Freud: What does this mean?
Think about Wikipedia, what some [who?] consider the most complete general survey of human knowledge
http://en.wikipedia.org/wiki/Wikipedia:AWW
Isn't the wireless access is for that purpose???
So, I'm reading here that they convert the XML into proprietary metadata and compress that.
Why not use EXI (Efficent XML Interchange) http://www.w3.org/XML/EXI/ which has been tested as more efficient that gzip and requires less memory to parse? Especially since the XML processing can remain the same, since the nodeset is the same.
I go to Wikipedia for two things; when I want summarized descriptions of fictional stories, and information about weights and measures and official standards. And boring details nobody has any opinion about which are so chipped in stone, I don't have to worry about how fractured and tangled up the editors' various emotional/intellectual states were at the time of the entry inclusion. Stories are safe because they're not real, and it's hard for Official Culture to get along when it distorts the way you convert miles to kilometers. (Though it would bloody do so if it thought it could get away with it!)
Other than that, Wikipedia is Humanity's dream-state navel-gazing summary of what it WISHES were real and not what IS. There is so much content on Wikipedia I disagree with that I barely even notice the red-light flashing in my brain when I scan any given article, or as more often happens, the lack of an article.
No wonder you can cram that thing into 8 gigs. Hell, I was flipping through an encyclopedia from the 40's, and under "Dynamite", it had detailed instructions on how to MAKE it yourself and how to blast rocks from your property. --Wikipedia isn't even as informative as a general knowledge set of dead-tree books from last century. But it IS smaller and more portable. I guess that's good.
And sorry, XKCD fans, but the Hitchhiker's Guide, (fictional though it is), isn't hamstrung with political correctness and facts washed out of existence by human insecurities. Maybe when humanity grows up and can handle looking at reality straight on will Wikipedia become something to be proud of. At the moment it's merely the litmus test for social maturity with embarrassing results.
Sorry for ranting, but honestly!
-FL
you can install it over wifi or from iTunes over USB, retard.
Do you even lift?
These aren't the 'roids you're looking for.
http://www.instructables.com/id/SBK1NAUFF78M26B/
I found these instructions in May 2008 and created a reasonably current snapshot of wikipedia that is still rather compact on a Psion 5MX. Not quite the same "curb appeal" as an iPhone, but a lot more functional.
Best,
Bah, that's nothing. I made an offline Wikipedia midlet! Unfortunately J2ME is unpleasant to say the least, and my phone only supports 2 GB SD cards so it only has some of the articles and without text.
I've been using this app for quite a while on my 1st gen iPod Touch, and it works and works well. It's amazing just how many articles it has. Other than some cosmetic and minor feature issues, the only real limitation is that Apple limits data file size to 2GB, so there is an obvious limit as to how much can go into the file. But it is amazingly complete. No images, no fancy tables--just text articles at your fingertips.
If you Jailbreak your iPhone/iPod Touch, then an excellent alternative is the Wiki2Touch app. Unfortunately, it seems that it's been pretty much abandoned in development, so it may be hit-or-miss if it works on OS v3.x. This implementation was REALLY slick. It provided a 4GB data file (that was much more complete) and a small Web server. You enabled the Web server, fired up Safari, and pointed it to a local URL. The app presented quick and very readable articles. And if you went to the trouble to download and process, you could also add about 4GB of image files to make things more complete (on a larger-capacity device, of course.)
Here's a review that I posted for both apps just over a year ago on my iPod Touch Tips site:
http://jimstips.com/ipod-touch-tips/ipod-touch-review-wikpedia-on-your-ipod-touch.html
In both cases, the main complaint is updating. In order to update the data file, you have to re-download the data, and depending on the app, you are typically at the mercy of the developer to provide an update. Otherwise, you had to download, index, and install the HUGE files yourself.
If you absolutely HAVE to have updated, offline data, check out the Wikipanion app. It's a nice compromise.
My mom always said, "Jim, you're 1 in a million." Given the current population, there are 7000 of me. God help us all!
If you have a PDA running Windows Mobile, Android or Blackberry, you can use WikiPock: http://www.wikipock.com/
Sounds a bit small.
---- Booth was a patriot ----
I use 2 free apps that let me access wikipedia. Nothing really new or radical about this app
Except it works when you're away from a hotspot, even if you paid only $220 (not $100 + $80/mo * 24 months) for your device.
Hell, I was flipping through an encyclopedia from the 40's, and under "Dynamite", it had detailed instructions on how to MAKE it yourself
Wikipedia doesn't have how-to guides. If you want that, use Wikibooks.
I think you could probably get answers to your questions by visiting public libraries, and talking to people. Maybe the "talking to people" bit might not get you definitive answers (though probably as good as a lot of Wikipedia content) but you might have found out a whole lot more. Also the public libraries probably had a lot of this info if you were looking for solid facts.
I appreciate a portable + off the net wikipedia would be a cool tool as well but nothing beats chatting to the locals.
You, sir, are a troll, so I shall not correct your ignorance.
Actually the app is 0.1MB from the app store, then it downloads the database from a non-apple server. And like the other poster said, you can install it over WiFi.
You should study how mistakes are made a bit more. When typing fast, I often mistype one work for another. Even though I know very well the rules for "its" and "it's" (a mistake you made in your post), "their" and "there", "than and then", sometimes my fingers just decide to type one thing even though my brain is thinking another. Sometimes I type "you" instead of "your", too. Or "to" instead of "too". None of this has anything to do with not knowing the rules. It's partially about not proofreading (c'mon, it's just a slashdot comment), but even proofreading can miss them because the brain sometimes reads what it means and not what it sees.
Moderators: Before moderating a comment Insightful/Informative, check to see if a child post has already refuted it.
App Store purchasers of Collison's open source application can browse and search the full text of Wikipedia when stuck in a plane
This page is not recommended when you're stuck in a plane...
How did you get the current snapshot of Wikipedia? I'm working on the Mark 2 (Nokia N770, shinier hardware but sadly hamstrung by the 2GB memory card limit) and I want a current version of WP, not some years-out-of-date CD edition. If you can provide some details I'll put them in the Instructable and give you credit.
- PKM
I've always thought somethings like this would interesting for a standard operating system, for when your Internet connection goes down, for when you're traveling with a laptop and don't have access to a wifi network, or, in tough economic times, for when perhaps your financial situation declines and you can't afford Internet access (hopefully temporarily) but still have the computer you bought when times where better (or were given an old legacy computer by someone no longer using it).
It's kind of a pet peeve of mine to see so many things created for the closed relatively small i-phone ap store and not ported to the operating systems we all have for normal computers. It's not an issue with Apple, I love itunes and ipods, it's just that I'd like to see these things available to a larger market, maybe as open-source aps. I've actually been waiting on something like this for a long time, specifically for Wikipedia, but I don't have an iphone or an ipod touch, and a monthly fee is a bit much for something that is a essentially a "backup" Wikipedia.
I bought this application 6 months ago and there are 3 majors problems with it:
1) The search function is broken because you need to type the exact word (prefix)
2) This is plain text: no pictures and no tables so most articles with "list" are useless
3) No update mechanism so the dump used will be outdated soon.
Is there a version of this that will run in a web browser? Anyone have a link?
Who would win this election: Andrew Weiner vs Andrew Weiner's weiner.
I had this for ages with Windows Mobile had this for ages with Tomeraider3 wikipedia database. Instead of using bzip2, it's more efficient and elegant to store the table in sqlite and use it's excellent sqlite3 compression.
You can download the database in a variety of formats. You'll need to trim it down quite a bit; the download of just the article text (no images, no history, no user pages, no discussion) is over 4GB compressed. You can probably compress this better with something a bit more domain-specific than gzip, and you can probably eliminate a lot of articles that are just stubs.
I am TheRaven on Soylent News
Is this an ad ?
try this link from your mobile phone:
http://wapedia.mobi/en/
That way you get the whole thing, up-to-date, and with no trouble or major memory usage.
I know you're being funny, but my first idea for how to implement this would be to
I use a browser for viewing /usr/share/doc/**/*html. Not all uses of a browser have to leave 127.0.0.1.
Wikipedia definitely wouldn't get past the Apple censors. Certainly not with images like this:
http://en.wikipedia.org/wiki/Gokkun
http://en.wikipedia.org/wiki/Bukkake
http://en.wikipedia.org/wiki/Human_feces
http://en.wikipedia.org/wiki/Meconium
With North Korea's nuclear weapons threats looming over the whole world every day, one must wonder what would happen if, one day, nuclear missiles are fired, plunging the world into a post-apocalyptic darkness.
The progress of humanity could be lost with the destruction of the Internet, libraries, etc. Luckily, you can now carry the history of the world and beyond - on your iPhone! Combine that with a power generator, and you'll still hold the history of the world!
TiniWiki also does this. I haven't done a detailed comparison with the one in the article, but I last time I looked TinyWiki was pretty good. They had two advantages over some other similar products: a) they had more of Wikipedia, not just a cut-down or old selection, and b) they could do incremental updates.
I have used both Patrick's app and this other one called TiniWiki.
TiniWiki is WAY BETTER! Patrick's app doesn't even know who Obama is!!!!
I couldn't even do proper search on Patrick's app and it does not have ANY references and info boxes, etc. All the important information in a wiki article.
Disclaimer, I am a friend of the developer of TiniWiki, but I have used both so I can say I am very impressed with TiniWiki.
And they support updates!!!!
I heard these guys http://www.mobilewikiserver.com are looking at doing that at some point
The article is clearly about an iPhone.
aarddict can already do this on the nokia internet tablets (and it works on linux and windows). there are dictionary files available all the major languages. http://aarddict.org/
Imagine the archeologist who finds it 1000 years from now. Awesome snapshot of this moment in history and whoever buries it, will be remembered for centuries after it's found.