Linux and OSS to Aid the Library of Congress

flakeman2 writes with a link to Linux.com article about Linux's new role at the Library of Congress. The national archive of books is looking to begin an ambitious digitization project, aimed at getting some rare and crumbling documents into the public record online. These will include "Civil War and genealogical documents, technical and artistic works concerning photography, scores of books, and the 850 titles written, printed, edited, or published by Benjamin Franklin. According to Brewster Kahle of the Internet Archive, which developed the digitizing technology, open source software will play an 'absolutely critical' role in getting the job done. The main component is Scribe, a combination of hardware and free software. 'Scribe is a book-scanning system that takes high-quality images of books and then does a set of manipulations, gets them in optical character recognition and compressed, so you can get beautiful, printable versions of the book that are also searchable,' says Kahle." Linux.com and Slashdot.org are both owned by OSTG.

Comments

Help the Library of Congress save American History

[JanusFury]

For the past few months Microsoft has been dispatching crack teams of special operatives into the past to alter the course of American History for their benefit, in hopes of eventually transforming the United States into the New Microsoft Empire. But little do they know, a world-weary Librarian and Ex-Marine at the Library of Congress won't stand for that shit. He's put together a team of agents in hopes of reversing the damage to the timestream before it becomes irreversable. Together with Agents Linus Torvalds (Technology Specialist - Special power: x-ray glasses), Donald Trump (Logistics Specialist - Special power: nuclear fusion comb-over) and Stephen Hawking (Quantum Physics Specialist - Special power: medusa glare), he just may be the only hope for American History's future.

more info

[wwmedia]

more info in this interesting article http://www.nytimes.com/2007/03/10/business/yourmon ey/11archive.html?ei=5090&en=9bf0874841a9d705&ex=1 331182800&adxnnl=1&partner=rssuserland&emc=rss&

Re:Hmm...

[rednuhter]

RaTFA (note the lowercase "a" for "all")
"the Internet Archive has migrated Scribe entirely to Linux, and Windows support has been dropped."
Seems focused on Linux to me.

All copyrighted works should be held

[rtb61]

They should expand the role so that all copyrighted works are held at the Library of Congress. It would certainly save the confusion over who holds what rights to what content. At the same time, unless congress wants to hold and distribute material of questionable moral quality, the copyright law could be amended to limit the protections of copyright to those works that do actually further the arts and the sciences as defined in the constitution.

The revisions to the law would not be infringing freedom of speech, in fact by allowing the free copying of works that did not further the arts or the sciences it would be limiting copyrights impact upon the freedom of speech. If people are really concerned about the quality of content, they should remember that eliminating the profit motive will have a substantial impact upon the amount of questionable content that is out there including movies, music, pictures and literature. Most of the members of the RIAA and the MPAA have a total disregard for the harm their content cause to society, let them feel some of the pain, wipe out the copyright protections on some of their more divisive content ;).

Re:All copyrighted works should be held

[jimicus]

At the same time, unless congress wants to hold and distribute material of questionable moral quality,

Stop right there.

When the purpose of your organisation is, to put it in very simple terms, "catalogue everything", you can't start making exceptions on moral grounds on the simple basis that what constitutes "questionable moral quality" today may be totally different tomorrow. Furthermore, who gets to define "questionable moral quality"? The closest anyone's ever come to creating such a definition is to say "Well, I can't actually come up with a concrete definition but I knows it when I sees it".

The most important part is not free software

[Ed Avis]

As the article says, the OCR itself is still done with proprietary software. I wonder if Google is using Tesseract for their digitization efforts. It would be cool if the original raw scanned images could also be archived and available for download - then you could print your own copy of the book, check the OCR for errors, or even do some weird genetic algorithm thing to make a LaTeX style that typesets the text in the same format as the original book.

The sad part of digitization.

[Lethyos]

Eventually we will have no physical record of these writings and may someday learn from the digital copies that Benjamin Franklin, George Washington, and others had offered enthusiastic support for wiretapping and other forms of electronic surveillance.

Quality as well as quantity, please

[Ankh]

The books I've looked at have been scanned at a resolution that's more or less adequate for OCR, but isn't really adequate for reproducing fine woodcuts, and is hopeless at metal engravings. I've found from my work on fromoldbooks.org that anything less than 1200 dpi generally produces pretty poor results for images, so that, for example, you can't read the signatures of the artist and engraver, still less compare engraving styles. It would be sort of like having a paraphrase of the text instead of the actual words.

It does, of course, vary a lot depending on the style of image. Bold illustrations for children's books, for example, do better at, say, 800dpi greyscale or colour. Fine steel engravings with lines at, say, less than a tenth of a degree from horizontal (they were done by hand after all) and that come out only a couple of pixels wide even at 1200dpi just turn into gray mush with weird banding artefacts until you go to a higher resolution (I use 2400dpi). There's a widely-cited study indicating that an "ultra-high" scan resolution of 400dpi is more than sufficient, based on an extremely small sample of images.

The damage that's done by poor quality digitization is that it makes it harder to justify doing a better job in the future.