Ask About Open Source Online Info Resources

This is a "double header" interview. Our guests are Jimmy Wales of the recently-started Nupedia open content encyclopedia project and Michael S. Hart of Project Gutenberg, which Hart started in 1971. The two projects are very different -- Nupedia is creating an encyclopedia, while PG is creating an open-ended database of public domain and out-of-copyright texts -- but they are similar in that both projects' primary goal is free (beer and speech) access to information. Post questions (one per post, please) below for Wales and/or Hart about their creations (or any related topic). We'll send 10 of the highest-moderated questions to them tomorrow, and will publish their answers as soon as they get them back to us.

Comments

Encyclopaedias are obsolete

[Chuck Flynn]

There's nothing in the world that can't be learned from a quick search via google, be it bomb recipes or the correct spelling for "partner". As search engines become more complex, directory listings (especially human-crafted ones) will become increasingly irrelevant.

What matters now is not where your data are hosted, but how others can access them. That's why it's far more important to get government-subsidized network access for our schools than it is to give them physical access to dead-tree books. Textbooks are just an excuse to milk readers out of $100 and rising, and I won't be sad to see them go.

Re:Encyclopaedias are obsolete

[JCCyC]

Good. You touched a subject that never ceases to nag me in the back of my head: textbooks. So here's my question to the interviewees:

What do you think of the idea of Open Textbooks? For example, books on World History, Biology, Math, Physics etc. that can be used in high schools and for which no copy restrictions are in place? Schools and/or parents and/or students would be able to print the book themselves at a fraction of the cost. Maybe the result wouldn't be so nice-looking, but it would be effective.

Think schools in poor neighborhoods, or in the Third World. Think cheap, fast inkjet printers. Think a central repository (or a number thereof) whose contents is certified as "Good For Schools" by some reputable academic body, govt-ran or not.

Re:Encyclopaedias are obsolete

[KjetilK]

There's nothing in the world that can't be learned from a quick search via google, be it bomb recipes or the correct spelling for "partner"

Here's the problem: You are looking for the correct spelling for "partner", and you find a web site claiming it to be "parttner", another "pratner", another saying "partner", so who do you trust? How do you figure out what the correct spelling for partner is? You'll probably turn to authorities from the dead-tree world, so the question is "how do you build that authority on the web?"

It is even more true with bomb recipes sites, they are more likely to earn you a Darwin Award than make you an effective terrorist...

So, there is a lot of room for humans still. To some degree you can rely on author-provided metadata, but only as long as you can take out spammers, first-posters, trolls, etc. And, given that lameness filters take out no more of those than censorware takes out pr0n, it'll always be a place for reviewers.

Now, what the Nupedia folks is trying to do, is to create an environment where review is rigorous, on the web. You won't have to turn to dead-trees authorities to get reliable answers, the reliability lies in the review process of the *pedia, and I say that as more and more information becomes available, third party review will become increasingly important as you really don't know what to trust...

Now, Nupedia may not be an encyclopedia in the classical sense, then, but I say a centralized source is still a Good Thing [tm] for many years to come.

Opposition from the 'For Pay' industry?

[Bonker]

Have you had any overt opposition from the 'For Pay' publishing industries? If so, what is it like. Do you expect legal challenges?

That's a lot of dough.

[laserjet]

From the FAQ:

Project Gutenberg began in 1971 when Michael Hart was given an operator's account with $100,000,000 of computer time in it by the operators of the Xerox Sigma V mainframe at the Materials Research Lab at the University of Illinois.

Anyone know how much computer time that is, or how it was computed? Unfortunately I wasn't alive at the time, and it seems like a weird way to measure time. But if time is money, how much time is 100 million dollars?

How long should copyrighted work remain copyright?

[DoorFrame]

Here's the question (which I'll follow by a little discussion of my one) but basically the point is how long do you think books, or other content, once given a copyright, should be able to maintain the exclusive rights to their work? It seems that books and music never enter the public domain anymore, why is this?

Anyway, here's some discussion (not part of the question and doesn't need to be forwarded):

Ok. I look at this situation and I'm torn. On one side is the fact that without copyright and trademark law, there is little or no incentive to create new works of art. On the other side, with trademarks and copyrights, we're living in an anti-capitalistic world meaning resources are being wasted.

So, we come to a middle ground.

The real problem, as I see it, is not that there are currently songs which are copyrighted and cannot be copied and sold. This is only fair and I, along with most other producers of content would be annoyed if, say, I wrote a book and tried to publish it only to find the next day that another company was giving a reprinted copy of my book away for free as a grab bag prize. Suddenly I cannot make a living (there are no performances for authors). So I do need copyright.

On the other hand, the blocking of others from selling something that they can produce and distribute more cheaply than I can is a shame, and there is no reason that after I'v been allowed to make my money from a product someone else can try to do a better job selling.

What we really have is a problem of time. For some reason it seems that copyright, unlike how it was originally intended, no longer expires. Patents on medines run out after 7 or so years to allow generic copies to be made, why can't the same hold true for content?

So, after 10 or so years, all content enters the public domain and can be reprinted or resold by anyone who wants. Anybody today could print up and sell Beatles albums at whatever cost they decided to charge. Suddenly there would be a true free market for Beatles recordings and the market would decide the price, instead of one company in an artificially controlled pricing system.

That's just my idea, but I truly believe it should be applied to all content: movies, music, books... everything. Give the authors ten years to make thier riches, then give them to the masses to use and reuse as they see fit. The public domain would once again be bountiful.

Technical overview?

[small_dick]

Hi,

I wrote some optical storage/doc scanning code in a previous life...would you be willing to share some experiences and insight into the mechanical/database/indexing side of things, past present and future?

problems?

[Xeo2]

are either of you worried about possibly erroneous submitions, whether it be a made up encyclopedia article, or badly translated public domain texts? In addition, what will the final forms of both of your products be? CD/DVD or internet? If internet will there be some kind of registration required?

How do you handle "Free" information?

[nicholasperez]

Information is always documented in some form or another. How do you plan on creating this database of info without giving credit where credit is do? And usually that involves money to the parties in which you "borrow" information from, especially in a publication.

___________
I don't care what it looks like, it WORKS doesn't it!?!

Re:How do you handle "Free" information?

[Jeremy Erwin]

Citing sources does not imply a commercial exchange. Facts alone cannot be copyrighted, so when the new Encyclopedia publishes a value for the "gas constant" (R), it may choose to note the scientific journal in which this value was published, but it is not obligated to pay a "licensing" fee.

Intergration of the two projects?

[Squirrel Killer]

To Jimmy Wales: How tightly to you see integrating Project Gutenburg's materials? Will you cut-and-paste sections from PG into Nupedia? Will the entry on Shakespere link straight to PG's texts of his work?

To Michael Hart: I'm well aware of your desire to keep PG e-texts as clean ASCII with nothing linking to other projects and the like, but would you link from the PG website (not the text themselves) to the Nupedia project? As in the previous example, while brosing the various Shakespere works, will I see a link to his biography on Nupedia?

Personally, I think that this kind of intergration is what will really add to Nupedia, as well as giving PG more value in that you can easily find out more about an author. I had been thinking about doing something like this, but just haven't had the time to do it right or the self-confidence to release what crap I did have to the outside world. Without biasing you answers, I really think that this kind of intergration would really be a boon to both projects, and show the benefits of open projects working together to create something greater than the sum of their parts.

-sk

Top-10 Copyrighted works you want if you could.

[DG]

For Mr Hart:

If you could pick any 10 currently copyrighted works, and have them placed in the public domain (specifically for inclusion in Project Gutenburg) what would they be?

DG

using text in other works

[po_boy]

It is often claimed that GPL'd code is not used in some projects because it would force the authors of the project to be more open with their code then they would like.

In short, I would like to know how you two believe this concept carries over into the content world. Is their an analogous effect, and is this type of work better or worse off than software in overcoming this effect?

More specifically, I see that the works in Project Gutenberg are primarily (all?) public domain, so they may be referenced, altered, and distributed in quite a few ways with few problems. The content in Nupedia, however, is held under a licence more like the GPL. Do you feel that this restriction will cause that content to be used less by people since it would place restrictions on the way in which they could release and distribute derivitave works? As the amount of content released under the Gnu Free Documentation Licence increases, do you think that it will have as easy of a time becoming accepted and used as software released under the GPL, or do you think that the restrictive nature of the license will have a more deleterious effect on the works released under it?

continue

[pouwelse]

A question for both the projects Gutenberg and nupedia:

Do you think your project would thrive and survive if you were hit by a train, or is the project still very much depending on you for expansion and future direction?

Johan.

Commercial offshoots

[CarrotLord]

A question for both gentelmen: Is there likely to be commercial offshoots of the Nupedia and Gutenburg Projects, similar to the way the various Linux Distributions have grown from Linux and GNU? Are there any ways planned or envisaged for companies or individuals to profit from these open projects?

PS: note that I consider profit a good thing in general, and this is not a troll or trick question. I would like to see profitable businesses built on the free exchange of knowledge.

rr

Project Gutenberg acceptance in schools

[Mendenhall]

I ran across a very interesting phenomenon recently with Project Gutenberg and the local public school district. My son needed a copy of "Plunkitt of Tammany Hall" for school, and it was not available without a long lead time from bookstores. I looked at Gutenberg, and found it, and printed him up a neat copy. I also printed an extra for him to give to his teacher, so students could copy it and not have to buy the book. I made it quite clear to the teacher that this was a legal operation, etc. However, my son says the teacher shelved the copy, and indicated little interest in providing it to students to copy.
It seems that free texts such as this would be the perfect thing to use in history courses, where students often buy a book, read it once, and never use it again. School systems could save the students a _lot_ of money this way, and with very little effort on the part of the teacher. Many copy places (such as Kinko's) even will handle distribution and sale of such copies to students, with no effort on the part of the teacher, and a lot more cheaply than buying a book for one use.
Do you have any idea how to convince school systems of the value of this approach? Given the large number of historical texts available, it seems that it would open the doors to teachers use of a lot more original material in classes without much effort or expense.

Appropriate Copyright Length?

[coldmist]

Originally, copyright length was 12 years, with the option to extend it on the last year for another 12 years.

Currently, it's up to 95 years (if memory serves).

According to the Constitution, it was supposed to be for "limited times," but 95 years is longer than most people's average lifespan. To me, it seems that the copyright protection is effectively "forever" since odds are an average American would never (legally) get the chance to apply creative talent to make a derivative work from the Star Wars universe, for example.

What do you consider to be an appropriate copyright length, balancing the need to pay content creators, versus the Public Domain and society's claim on it? And, if you think it should be considerably less than it is now, how does the US's Berne Convention agreement effect/influence what can be done?

Re:Appropriate Copyright Length?

[Hard_Code]

Obviously the question is posed to someone more knowledgable than me, but my opinion (for what it's worth) is that "limited times" needs to be tied to the speed of change in the market. Sure, perhaps long-lived texts, or pretty mundane inventions can do with a pretty long limit...because there isn't much need to foster "incentive" in those markets. However a year in the software industry might as well be a decade in some other industries. Here, copyright, and patents, need to be proportional to the speed of the market. How can we possibly incentivize creativeness when we have copyright and patents that expire long after the idea or product has become obsolete? I don't think there is one, final, magical number that will work for everything. We just need some sensible legislature, and patent officers who have real brains, instead of being driven by silly throughput or profit motives (I put through 100 entirely frivolous patents this year! Gimme my raise!).

Project Gutenberg file format

[rodentia]

I have been an avid fan of the project for as long as I've been aware of it. My question has several parts pertaining to presentation technologies.

We're a long way from 1970 when ASCII was the only viable lingua franca for a network; is there any discussion of updating the file format for the project? Specifically, something *ML-ish which would allow for presentation in multiple output formats. I am thinking of the spread of e-book readers and the like and increasing the potential readership. With a proper infrastructure, project texts could even be rendered to adaptive browsers with VoxML or other technologies.

Secondly, if the project doesn't choose to modify its longstanding ASCII formatting standards, are there efforts afoot to programmatically apply some structured tagging on-the-fly to allow for easy translation by other tools? Is this an itch I'll have scratch for myself?

Re:Project Gutenberg file format

[Eccles]

PG currently only requires ASCII text. Doesn't that make dealing with foreign languages difficult? Is there a Unicode format that might be more appropriate, at least for foreign language files? Also, it seems slightly incorrect to say you're not using a markup language, when in fact you do use a trivial one; using blank line separators to indicate chapter and other breaks. Wouldn't the creation of a trivial markup language (or now, selecting a simple subset of HTML) be worthwhile?

The Bono Copyright Protection Act

[jonathansamuel]

How damaging has the Bono Copyright Protection act been to Project Gutenberg?

This idiotic piece of legislation retroactively increased length of copyright protection for works written in the 1920s, so that Robert Frost's poetry and many other works will now be kept out of the public domain for another generation.

Is there any possibility this act could be repealed?

Not:Encyclopaedias are obsolete

[rodentia]

As a history buff, I can assure that there is a large universe of information which is not now, and never will be, available from a quick search via google. There are not enough trained monkeys out there to get the vast majority of pre-digital texts onto the web.

That said, a good HTTP-based encyclopaedia is an almost perfect application of the technology and directly in keeping with the spirit of the original encyclopaedists: Voltaire, Diderot, et. al. It may be said that the project of the encyclopaedia defined the spirit of the Enlightenment: all human knowledge will at last be encapsulated within the covers of a book; available to ready cross-reference.

For Mr. Hart

[ContinuousPark]

I've noticed that the Project Gutenberg site has a rather straightforward interface, you get the database queries you need but I've noticed that's not very friendly for some users; computer illiterate users that I've recommended your website to and children, for instance.

I've also noticed that all texts are available as text-only and I understand your decision behind this.

So, my question has two sides: Are there any plans to build a front-end for PG that is more user-friendly; by this I mean, for instance, profiles of major authors and new acquisitions, featured writings each week, a section for children, personalization features so that the site recommends books for me, and so on. Are there any plans to, while always having text-only versions, also have automatically generated versions in other formats (pdf, postscript, and especially some of the new formats for eBooks or PDAs)??

I think some of these changes, just having a front page that changes everyday with new reading suggestions and lures the visitors to go and read (in the same fashion that makes people go to BN or Amazon to buy books) could make your site much more popular than it already is but how high is this on your list of priorities, if at all?

ps. Kudos on the excellent work you've done through the years!

Mutopia

[Uruk]

Hey guys, on the free (beer/speech) information range, let's not forget:
Mutopia.

Sonny Bono Copyright Term Extension Act

[yerricde]

It appears copyright has been continuously extended on everything produced since 1917 (not sure that's the right year, the copyright office website has it somewhere).

The correct year (in the US at least) is 1923. To me, a copyright date provides only one bit of information: before 1923 or on-or-after 1923. Read my short essay on the Sonny Bono Copyright Term Extension Act, the root source of DisneyCo's power.

I would gladly donate to an organization doing copyright law reform.

You could start at the Eldred v. Reno page. Also consider joining the Electronic Frontier Foundation.

Like Tetris? Like drugs? Ever try combining them?

25 years tops

[yerricde]

I agree that the Bono Act has gone too far. Imagine if patent terms were as long: we wouldn't have generic drugs at all. IMHO, 25 years is more than enough to cover an author's expenses and provide a decent profit. If 17 to 20 years for a patent is enough to keep the drug industry running, why wouldn't a similar time period also work for the copyright industry?

Like Tetris? Like drugs? Ever try combining them?

beyond file format issues

[thex23]

What is a document when it is stored electronically? How long does it last, and how do I know it is valid?

There are a lot conceptual and practical issues to be dealt with when designing an information resource that is supposed to have value over an extended period. In this vein, what efforts have been directed towards understanding strategic issues like:

• authentication and security
• correctness of content and granting of "authority"
• stability of media and life-cycle/maintainance issues
• writing style, data structure, and presentation (w/ technologies like PDF, CSS, etc.)
• use of "plain language" for comprehension and translation across cultures now and in the future

We thieves, we liars, we vandals, and poets. Networked agents of Cthulhu Borealis.

GNUPedia vs. Nupedia

[dsplat]

Is there any relationship between Nupedia and GNUPedia? Are these simply two independent implementations of an idea whose time has come? If they are independent, will they merge or continue separately serving different goals?

Can a public domain encyclopedia help?

[versimilidude]

Can these two projects be combined? There were quite a few encyclopedias published in the first part of the 20th century that are now in the public domain, and many of their articles are still good and useful. Actually, for many figures of the 19th century the biographies are better than what is published in modern reference works. Descriptions of, among other things, basic algebra, geometry, ancient and medieval philosophies are still valid. Of course entries like "Germany" and "Argentina" would need updating. Would the weeding and editing be more work than the final value of having a comprehensive set of listings?

Security Archive

[h8macs]

A real, unfettered, complete archive of security related material. From the most elusive to find to the current day featured on 'slashdot' . By materials I mean, books, how-to's, FAQ's that have long since dissappeared. One might argue that there is no reason to study old fixes, old rules, old policies. I have a different opinion than that, and would love to see a huge gathered archive.
Even if just a hyperlinked list.....Preferrably searchable!? ;-)

What exactly is your focus on this 'free' information?

:-( --- argh. Despair, I owe again. :-b

open source OCR software?

[Dr. Tom]

what are the best open source OCR efforts that you are aware of? G* and K* compatible scanner/OCR/packaging systems should be pushed to help get the most material online in the shortest time, using modern formats like SGML.

The Human Body

[Rick the Red]

How is Nupedia going to replicate those nifty transparancy overlays of the human body (you know, one for the nervous system, one for the skeleton, etc.) that World Book now does so well?

Are you going to turn your lack of a print version into an advantage by doing things the others (I'm thinking WB and EB, not Encarta) can't do on paper?

Help needed by Nupedia?

[bcrowell]

What kind of help does Nupedia need the most? --

• writing articles
• peer reviewing
• editors for certain subject areas
• copyediting
• software work
• software documentation
• ...?

Certain subject areas (e.g. physics) seem to have some people interesting in writing articles, but don't seem to have enough volunteers for the editorial jobs to get articles through the process. What jobs need to be filled in what areas? Do you think the editorial jobs are hard to fill because people don't want to make enemies? ...because people don't think they'd be fun jobs? ... because it doesn't scratch people's itch?

Re software work, Nupedia has just changed over to new software, which seems a little raw. Is it on CVS? Would you like to get offers of help from people with good track records writing open-source software?

Sorry if I sound like a whiner, but as a Nupedia newbie (nubie?), I was pretty discouraged by the lack of understandable, current documentation, and stuff like underlined text that turned out not to be real hyperlinks,...

The Assayer - free-information book reviews

iconicism and art

[funferal]

A project such as Project Gutenberg concentrates on the content of a work of Art, rather than the object itself (e.g. an original Shakespeare Folio).

The view has been expressed that as content becomes more freely available, the value placed on objects - such as first editions, or signed copies - will increase.

What does Michael Hart think? Will people be more inclined to buy books for their curiosity/iconic value? Will this reduce the value placed on the content, the thoughts and ideas, as people buy books much as they now buy antiques or works of art? Will this be a good or a bad thing?

Andrew Ó Baoill