Open Library Project Takes Flight

Aaron Swartz today announced the launch of the new Open Library project. The goal of the project is to produce the world's greatest library on the Internet free for anyone to use. Starting with the Internet Archive's book scanning project and organizing the insertion of new content via a wiki-type model the project seems to be off to a great start. The demo, source code, and mailing lists were all opened up today in hopes of drawing interest from the public at large.

Comments

Re:Awesome

[Creepy Crawler]

Well, you can thank extensive copyright for that fact.

Go Disney.

Relevance?

[cdrguru]

As long as it is limited to rather dusty tomes that are "out of copyright" this is going to have limited, if not zero, value to most people. What exactly is the difference between Open Library and Project Gutenberg? Aren't they going to have 99% overlapping content?

Re:In response to your question:

[jandrese]

I find it depressing that if someone came up with the concept of a free library system today, they would be sued out of existence by the book companies. What is perhaps one of the greatest triumphs ever for the poor uneducated masses would not stand a chance in our current legal environment.

Take flight?

[Anonymous Coward]

"Taking flight" normally denotes escape from a perilous situation, not emergence as is intended by the author.

Mod me down if you must but it's annoying when otherwise intelligent people cannot write a simple sentence and the editors are so lax in their responsibilities.

I must be new here.

Re:Project Gutenburg

[Anonymous Coward]

So, its like the Library of Congress that links to your Amazon Referral Page?

Talk about sleazy.

Not Project Gutenbeg

[krelian]

Don't compare this to Project Gutenberg. This is the supposed to be the Internet Movie Database" for books (as far as I understand anyway). Anyway, I am pretty sure that a big part of this information can filled with calls to Amazon web services.

Re:Not Project Gutenbeg

[dwarfsoft]

This is exactly what I was going to say. It is basically an overview of books with links to Amazon or wherever. I forsee reviews, quotes, and links on par with IMDB. Only there is a benefit of being able to have full-text books too.

I had a play with it and it is quite limited at the moment. I did manage to add a book, but there was minimal instruction on how to go about this, and uploading covers at the moment is not available (as far as I could determine in 5 minutes anyway).

Re:Awesome

[illegalcortex]

Yes, I particularly enjoyed Human Genome Project, Chromosome Number 08. Some fine reading there.

C'mon, I would be fairly disappointed with a library of 21,000 real books even if it contained only fiction from random authors from 1900-2000. Gutenberg doesn't even have that much depth.

That's not to take anything away from them. But to make claims about it being a good selection based on "21,000 - gee that's a big number" is a bit ludicrous.

A Library Card Tip

[Revenge_of_Solver_Ta]

This is great news, I hope it actually works. Related: I recently discovered my local library has about 50% of the books I usually buy. Why didn't I think of this earlier? Must of lost about $10K from that during the last decade. Now, if you'll excuse me, I must go check out a copy of "How to Make a Your Very Own Video Game in 16 Days Using ONLY...Wordstar!"

Re:Project Gutenburg

[PMBjornerud]

I find these scanned original pages FAR more restful to the eye than any other form of electronic book. This way, I can sit down and read a complete book on the screen -- without suffering the eye fatigue that comes from reading large swaths of ordinary onscreen text. I think it has a lot to do with print fonts being designed specifically for the eye, and somewhat to do with the normal yellowing of paper that produces a less glary background. This does not make sense. A scanned document will always have artifacts and imperfections from the scanning process and should by definition be harder to read. A well-sized font on a pleasant background should beat scannded text every single time.

Your issue is more likely that there are a lot of crappily designed webpages out there.

If you're reading "large swaths of ordinary onscreen text", do this:
- Copy-paste in into any word processor
- Choose a nice, big font. (Small is good for UI, not for 400-page-novels.)
- Use a dark background. A page reflects light, a screen projects it. You do not want glaring white.
- Use 8-10 words per line.
- Profit! Err... less mental exhaustation, at least.

Pay extra attention to words per line. It's a key reason onscreen text is often hard to read. Too many words per line, and you'll have a mental overhead every few seconds trying to figure out which line you just read and which is next. Basically, books do it right and you want to display onscreen text at a similar width. Scrolling is easy these days, and wide lines is a remnant from when computers required a click-and-drag to scroll.

Wide books and newspapers are divided into columns. There is a reason for doing this, but almost nobody seemed to think about that when they display text on screens.

Heck, even slashdot defaults to a glaring white background and text stretched all over my 1920 pixels. Go figure.

Re:In response to your question:

[fyngyrz]

Anyways, the good news is that libraries do exist, and aren't going away.

No, of course not, because they're protected by copyright law, which in turn grew out of article 1, section 8 of the constitution. Just there will never be a restriction on keeping and bearing arms... uh, oh, wait. OK then, like there will never be restrictions on speech... no, no, turns out there are plenty of those. Mmmm, ok, just like the feds can only take action on interstate commerce, because you know, that's an enumerated power they can't step outside... aw, no, they do that all the time. Well, it'll be like how they can't do searches or seizures without probable cause, oath or affirmation, and a warrant... oh... I guess that's no longer true. Well, of course they can't make ex post facto laws... except for the ones they've made, that is, you know, thinking of the children and such.

Wait. Why is it again libraries "aren't going away?"

Re:Not a problem for Pirate Bay?

[Anonymous Coward]

There's an assload of ebook torrents. What I want, and which I hoped just for a second this might provide, is a way to get access to science journals. The public is becoming more and more ignorant of science every day. The journals are locked up in ivory towers, public libraries usually don't have the funding to get subscriptions, and as a result every day more and more people are falling for faith healers and the like. I'll be able to pirate the next harry potter hours after it's released, if not sooner. JAMA, on the other hand, pretty much never shows up.

Kinakuta

[EnsilZah]

How about placing the servers somewhere where copyright law hold no sway?
Are there really any working data havens?

Some thoughts

[harmonica]

I know the project is just starting, but here it goes.

They should republish the raw data the same way Wikipedia and even IMDb does. I for one am not going to contribute to any data collection project that I can't later use myself.

Their schema doesn't differentiate between editions. If I understand it right, that means that for the 3000 existing editions of "Tom Sawyer" released over the years, by different publishers in different countries and languages, the book's description has to be replicated for each one. That can't be good. I don't have a quick solution to this myself. Sometimes (esp. with tech books), a new edition changes content significantly compared to the previous one, sometimes they're exactly the same.

Collecting the cover images is a great service. However, doesn't this infringe on the publisher's copyright? Is this still fair use? What about countries like Germany without fair use laws--will German books still be OK because the data is collected in the USA (I guess)?

Add a feature to upload book descriptions as XML. Suggest a DTD. I have a list of my book collection stored as an XML file, so have others (maybe not natively, but book collection management software usually has an export function). It should be possible to automate the process of adding book information already stored in some digital format.

There should be some category system to pick from. Some may put Tom sawyer into "Novel, USA antebellum", others into "Novel, USA 19th century".

Somehow connect this to Wikipedia. The more prominent books have article pages. Maybe data could be retrieved from it as well. There are currently Tom Sawyer articles in 16 or so languages.

The edit page should group items better: stuff everyone understands (year published, title) first, then those things only specialists know.

The edit page's descriptors shouldn't be images but text which links to an explanation page for the same reason. BISAC? LCCN? UCC13? I know, I can find out what those are with a search engine, but I shouldn't have to.

Prepare for i18n. I guess LCCN is a library of congress code number? Those types of libraries exist in other countries, too. Each book can have a gazillion codes. Make this another tuple in the database: (book_id, code_id, code_value) instead of (book_id, lcc_id, isbn10, isbn13, 10 other codes in the same record).

Also i18n: store language codes with all textual columns. A description is most likely going to be Hungarian for a book published in Hungary in Hungarian.

This complicates the schema a lot. Having very few tables is tempting, but it usually doesn't work well with the real world.

Re:Some thoughts

[Teancum]

Here is some additional food for thought about this idea.... coming from somebody who has only given this concept just a few minutes of thought, but having dealt with this issue extensively in the past (of trying to catalog e-books):

The kinds of skills necessary for doing actual cataloging work.... classifying and organizing knowledge... are so rare as to be a very precious jewel of a person if you ever do find somebody like that. And developing these skills is not something very easy to accomplish either. Certainly some basic tools can be developed that would make it a bit easier to climb up the steep slope of learning various cataloging techniques and understanding ontology as a discipline, but it is unusual. Most professional librarians that I have met (I'm talking people who actually work in real libraries) may have taken a college class or two about the subject, but even they seldom get into this sort of activity.

Here is the main point about this discussion, and why this is a much harder task than is apparent: Almost all cataloging work in the USA (and the rest of the world too, BTW) is done by the national libraries (aka Library of Congress) and the thousands of other libraries largely rely upon that cataloging effort to come up with their own numbering scheme. Especially with the "cataloging in publication" process where the process of formal copyright registration assigns cataloging numbers happens well before the book even arrives at a typical local or even university library.

At even a large library, those involved in the cataloging of content are usually a small team or even a single individual who has to catalog the couple dozen books that come in each year that aren't from major book publishers (often local histories that are self-published). Even then, it is hardly a full-time job and library staff like this usually have many other job duties.

How this relates to eBooks and content on the internet is that there are many electronic resources in book-like form that are largely uncataloged. I would put it at close to 100,000 books, perhaps even more that are original "books" that have been written in the past 20 years, and are available under a free (as in beer and freedom) copyright license. The "low-hanging fruit" is the Project Gutenberg collection, but much of that surprisingly has already been cataloged in more than one form. This is because they are older books and have been cataloged years ago. While there certainly is value in preserving older documents like the PG collection, there is so much more, and in many ways more relevant explicitly because it is up to date.

BTW, in response about the cataloging numbers, you can't simply assign a book to a single cataloging ID and expect it to work in every situation (without something incredibly complicated). Every classification system; ISBN, Library of Congress, Dewey Decimal, and about a dozen other classification systems; each have their own strengths and weaknesses. And different strengths and weaknesses. If a book has any value, it covers a very unique topic that is one of a kind, and it is these books which are the ones that you need to have a clean cataloging system that is able to allow you to "place" the book in a format that there are multiple methods for being able to find that content. For the hundreds of books about how to write HTML (to pick a topic that is common) they are largely the same... but my experience in trying to deal with book cataloging is that something so common like this is a rare situation, and at least 50% of all books in an e-book library are going to be something completely unique in terms of the topic covered. In short, you need the dozens of cataloging ID numbers for each book and not just a single cataloging ID number that is cross-referenced into a much larger and more complex database.

Change the name first

[Anonymous Coward]

A lot of the confusion here arises from the fact that this claims to be a "library". A library is where you can borrow books. An online library would be something where you can download books. On their site you can't even read books. It's a (bookstore/library/etext) catalog at most.

Re:In response to your question:

[OldeTimeGeek]

Wait. Why is it again libraries "aren't going away?"

Aside from the already mentioned fact that all books aren't digitized, it may be because Internet access is not universal, the barrier to access is still high (computers aren't free, right?) and one of the few places that you can get free access and access to a device to do it is, of course, a library.