Slashdot Mirror
Why Project Gutenberg Isn't There Yet
option8 writes "This wired article ('Any Text. Anytime. Anywhere. (Any Volunteers?)'), goes into good detail on why Project Gutenberg, and similar efforts, are far from creating a complete, free electronic library. A quote: "The mechanics of a universal library are simple. The tricky part: harnessing the free labor." Though it doesn't go into technology much, I expect there's a lot of potential in mass OCR tech and good speech recognition (faster to read a book aloud than to transcribe it correctly)."
That's not how project Gutenberg works. Most everything that's on PG is public domain - that means the copyright has expired. Thus, most of the stuff is over 70 years old. They didn't exactly use Latex back in the 1930s.
Besides, what I generally use PG for are the classics - greek/roman literature, etc... I don't think Plato used UNIX.
It's all got to be somehow entered from dead-tree-format copy. Currently, that pretty much means typing up the entire book.
--
http://nemilar.net - Not your grandmother's soup kitchen
That's pretty much it - most of the books are in the public domain. AFAIK, the rest are all donated by their authors.
From their FAQ:
What books will I find in Project Gutenberg?
We cannot publish any texts still in copyright. This generally means that our texts are taken from books published pre-1923. (It's more complicated than that, as our Copyright Page explains, but 1923 is a good first rule-of-thumb for the U.S.A.)
So you won't find the latest bestsellers or modern computer books here. You will find the classic books from the start of this century and previous centuries, from authors like Shakespeare, Poe, Dante, as well as well-loved favorites like the Sherlock Holmes stories by Sir Arthur Conan Doyle, the Tarzan and Mars books of Edgar Rice Burroughs, Alice's adventures in Wonderland as told by Lewis Carroll, and thousands of others.
These books are chosen by our volunteers. Simply, a volunteer decides that a certain book should be in the archives, obtains the book and does the work necessary to turn it into an e-text. If you're interested in volunteering, click here.
--
http://nemilar.net - Not your grandmother's soup kitchen
Distributed Proofreaders. Recently discussed on /. as well.
The requires LaTex source to post, which means a modern edition, which means it's copyrighted, which means you can't copy it (unless you have the publisher's permission.*)
Project Guttenburg only does texts which are in the public domain, which currently mean pre-1923 editions (PG of Australia has newer books) and, obviously, pre-1923 means that the only sources are print copies.
* pedantic point: it's the copyright holder's permission, which isn't necessairly the publisher, but usually is.
Keep in mind the following copyright rules:
1. Works first published before January 1, 1923 with proper copyright notice entered the public domain no later than 75 years from the date copyright was first secured. Hence, all works whose copyrights were secured before 1923 are now in the public domain.
(This is the rule Project Gutenberg uses most often)
Works published from 1923-1977 retain copyright for 95 years. No such works will enter the public domain until 2019.
2. Works first created on or after January 1, 1978 enter the public domain 70 years after the death of the author if the author is a natural person.
(Nothing will enter the public domain under this rule until at least January 1, 2049.)
3. Works first created on or after January 1, 1978 which are created by a corporate author enter the public domain 95 years after publication or 120 years after creation whichever occurs first.
(Nothing will enter the public domain under this rule until at least January 1, 2074.)
4. Works created before January 1, 1978 but not published before that date are copyrighted under rules 2 and 3 above, except that in no case will the copyright on a work not published prior to January 1, 1978 expire before December 31, 2002. If the work is published before December 31, 2002, its copyright will not expire before December 31, 2047.
(This rule copyrights a lot of manuscripts that we would otherwise think of as public domain because of their age.)
5. If a substantial number of copies were printed and distributed in the U.S. prior to March 1, 1989 without a copyright notice, and the work is of entirely American authorship, or was first published in the United States, the work is in the public domain in the U.S.
6. (This rule is complicated, and is seldom applied). Works published before 1964 needed to have their copyrights renewed in their 28th year, or they'd enter into the public domain. Some books originally published outside of the US by non-Americans are exempt from this requirement, under GATT. Works from before 1964 were automatically renewed if ALL of these apply:
At least one author was a citizen or resident of a foreign country (outside the US) that's a party to the applicable copyright agreements. (Almost all countries are parties to these agreements.)
The work was still under copyright in at least one author's "home country" at the time the GATT copyright agreement went into effect for that country (January 1, 1996 for most countries).
The work was first published abroad, and not published in the United States until at least 30 days after its first publication abroad.
This means that we can't simply take electronic versions of modern texts and put them in the archive, because only out-of-copyright books are in there.
The volunteer page is the place to start:
http://promo.net/pg/volunteer.html
No, the real REAL problem is that because of Disney, copyright lengths keep getting extended and extended. At the current rate, Mickey Mouse will never be public domain. This is actually unconstitutional, since Congress is enabled to grant exclusive rights for "limited times" only. But it's the way things are.
Most of the stuff on PG is public domain, IIRC. Unless Poe, Melville (I know it's wrong, so sue me), Shakespere, and others all climb out of their graves and form some kind of union (RIAA - Recently-undeceased Inkers of Aged Albums etc.) will people complain that they're getting ripped off by these works being put on the web.
Comment forecast: Bits of genius surrounded by a sea of mediocrity.
While this comment has been addressed, I'd like to point out that you can get pretty decent output from the Gutenberg texts by importing them into LyX. With just a little bit of work (basically setting up the chapters), LyX will allow you to create good looking PDF, Postscript, HTML, etc, along with the LaTeX source. Combine this with rbmake and you can even read them, complete with hyperlinks, on your eBook (if you have one!)
The article didn't say that OCR was faster than speech, it said that speech was faster than transcibing it.
Come on mod's, read more carefully.
poliglut.org: they're still alive and fighting the man
it is part of the philosophy of Project Gutenburg to publish all of their works in the lowest level stardard format, thus insuring continued cross platform, program independant readability, ad infinitum.
That means *plain* ASCII. Plain ASCII means you could read it in edlin if you really had to.
This is a Good Thing.
This also means that if you wish to format any Project Gutenburg text, in HTML or TeX for publication, you start with a blank slate and can immediately start to work your own will upon the raw text.
This is also a Good Thing.
KFG
While I like the project, I think the biggest problem is the interface to use the books. They end up in this crappy.txt format. The searching and browsing is slow and painful. If they just spent a little time on the website, they might get more support!
-- these are only opinions and they might not be mine.
#bookz on irc.undernet is an excellent place for ebooks, of course, with a little illegality behind it. Many of these are the same one's that have been floating around on alt.binaries.ebooks since the stone ages, but I think this unrestricted database is probably the best library created.
Hah, try transcribing "Huckleberry Finn", or any Dr. Seuss
No. Boycott Dr. Seuss. His estate submitted an amicus brief in favor of the Bono Act. Now that Project Gutenberg uses distributed proofreading, the Bono Act is the biggest barrier to the growth of PG.
Will I retire or break 10K?
Additionally translations might generate practical limitations. If a text was written in ancient Greece and translated to English or some other language in the 20th century, the translation might not be public domain even when the original work is. Of course you are free to read the original text or make a new translation. Anyway even if a piece of literature was public domain, the translation to your native language might not be.
That's exactly why. Since 1971 a wide variety of encodings and markup languages existed. 32 years later the only system still trivial to read is plain old ASCII. Project Gutenberg is most interested in preserving the texts themselves. The texts are quite well preserved in ASCII. Sure, some formatting is missing, but it's relatively minor for the majority of books in question. And given the existance of this unformatted text it's alot easier to create formatted text than from scratch, so you even get a benefit there.
I think you're a bit confused on semantic markup. By and large publishers aren't interested in semantics of the documention, just the formtting.
Search 2010 Gen Con events
That's basically what Distributed Proofers does. Except they OCR the book first, so the proofreaders just need to fix the OCR errors. Every page goes through two passes. Then the entire book goes into post-processing where a single person puts all the pages together, and checks for problems that the proofers didn't know how to solve (marked with an astrisk). Once Distributed Proofers finishes the book, they pass it on to Project Gutenberg where somebody reviews the whole text again.
Distributed Proofers currently has a problem. After the previous Slashdot announcement, they were overwhelmed with volunteers. The volunteers processed books so fast, they were running out of material to work on. Three or four people scan in most of the books. They have been slaving away trying to keep up with the proofers.
Distributed Proofers is also working on a standard to mark up the books to better preserve tables, illustrations, bold text, math, etc. I suspect that effort is being slowed due to the priority of keeping material on the site.
Gutenberg did NOT invent the printing press - He invented moveable type -a BIG difference
Before Gutenberg, there were printing presses, BUT you had to carve the master (the plate) for each page, and it could NOT be changed. Other folks had the IDEA of movable type, but what Gutenberg did was figure out a way to make it work (what he did was figure out how to make all the type the same length, so that when you press down, all the type comes in contact with the paper)
Movable type gives you one huge advantage - you can make up a bunch of sets of letters, and reuse them for many pages.
The total irony of this is that movable type is almost never used anymore - we make up a plate for each page. Of course, we are doing it with electronic movable type, but that is here nor there. Movable type started to go away with the Linotype machine - which made up one LINE of type at a time.
I think I still have an ingot of linotype metal around somewhere
-- 73 de KG2V For the Children - RKBA! "You are what you do when it counts" - the Masso
As much as I wish you were right, you're actually wrong on this. The Supreme Court ruled on the case, and found that what the Congress did was constitutional, and since the constitution grants the Supreme Court the right to interpret the Constitution, it is constitutional to do so. This will only change if the Supreme Court changes its ruling at a future date, or the Congress were to ammend the constitution to make it unconstitutional, this issue remains constitutional, as unfortunate as it is.
Well, until it's free, there's always textfiles.com.
Actually, a while ago I copied a lot of the Project Gutenberg library, along with some others, and created etext.textfiles.com.
In my experience, the reason a lot of people don't donate free time to transcription or other similar drudge work is because a lot of sites that encourage it steal it. Witness CDDB, and just wait to see how long before you pay for IMDB.
Lyx is nice, but I don't think that it really speeds things up. I can't imagine that Lyx could speed things up at all on a Tom Swift.
See what I've been reading.
The Great Library in Alexandria was a wonder of the ancient world until it got burned down as part of a domestic dispute between Mark Antony and Cleopatra.
Uh... what? For centuries people have blamed the burning of Alexandria's Great Library on the Romans, the Christians, or the Muslims, depending on which ones they disliked. But Mark Antony? Cleopatra? That's a new one. Maybe you're thinking of Julius Caesar, who gets the blame according to this fellow (a self-proclaimed Christian apologist).
I just found this site a few days ago. Essentially, volunteers can proofread one page at a time, so that huge time commitments of doing an entire book yourself are not required. Worth checking out.
http://texts01.archive.org/dp/
It might help to actually understand what you are talking about before you are so quick to dismiss it. DJVU does support searchable text, which can be inserted automatically via OCR. The advantage of this is that the OCR need not be 100% accurate to still be useful (vastly more useful and accurate than the indices in most books, for instance).
There is much pleasure to be gained in useless knowledge.
Also, there were prior claimants to the "invention" in Europe, such as Laurens Coster in Haarlem, Netherlands, and others in Bruges, Flanders (Belgium), Avignon (Waldvogel, who is recorded as having "steel alphabets" in 1444) and Bologna.
BTW Gutenburg's "invention" was not the length of the type. It was to have cast the movable type in metal using a matrix. As he was a goldsmith and his father was the Master of the Episcopal Mint in Mainz, this was a great instance of lateral thinking, adapting technology he knew well and applying it to a new field. He would have seen coins being minted and twigged that you could print books like that.
He also designed the press (adapted from existing wine presses) and came up with an ink that was suitable for the process of printing with this type of press (the ink had to be viscous, rather than the ink used for manuscripts).
His combination of the three things meant that he could successfully exploit printing commercially. So Gutenburg was probably the first to exploit it commercially, although he wasn't very successful (5 years (1450-1455) isn't a long time to have a revolutionary business). This fact has ensured that he is credited with the invention of modern printing.
I'll point out that at the end of 2000, there were only roughly 2000 etexts in the entire PG library (I copied them all to a single CD)... So if they're up well over 6,000, then they've made amazing progress in two years!
I recently bought a Franklin eBookman ($39.95 at CostCo!) and then, more recently, got an iPaq through work. The last five books I've read have been on one or the other PDA's (I had the Baen CD-ROM of Honor Harrington books and others). It still isn't quite as good as a paper book, but it is the best way I've found yet to read in bed.
:-), and the formatting leaves a lot to be desired. On the PDA's, weird page and line breaks or even bad justification or extreme ragged edges, are very disconcerting when reading.
I've been using the Mobipocket reader on both devices and the autoscroll feature is really cool -- you can prop up the device, turn on the backlight, and adjust the autoscroll to your reading speed. Hands-free, no reading lamp, no cramps from trying to prop up and turn pages.
On thing that strikes me is how much typography and formatting matter, which is, as others have pointed out, the problem with Gutenberg texts. I have read quite a few PG texts in the past (or at least used them for reference when I was looking for particular quotes or need a big text file to test something
-- stream of did I lock the front door consciousness