OpenDocument Alliance to Fight Digital Dark Age

OSS_ilation writes "A consortium of vendors and academic institutions -- including IBM, Sun Microsystems and the American Library Association -- has announced today that they are forming the OpenDocument Alliance as part of an effort to promote open file standards worldwide. The group will support the one truly open standard file format, OpenDocument, which is an XML-based file format used saving and exchanging editable office documents such as text documents, spreadsheets and presentations. Sun's Simon Phipps said he believed ODF would allow future generations to view all of today's digital docs and prevent a digital Dark Age from occurring."

Comments

Dark age already upon us

[saskboy]

The dark age has already happened several times. There are oodles of media formats from the 70's and on that are no longer readable today in the standard computer. Heck, new computers don't even come with floppy drives for 3.5" floppies. I hope they have a strategy to tackle media problems along with file format compatibility, because the medium is the message.

Re:Dark age already upon us

[dylan_-]

I know, all of my data stored on my Commodore 128 can't be ported to my Linux machine since it doesn't have drivers for the 1571.

Actually, this page has exactly that.

Cool name!

[Rob T Firefly]

Naming your interest group "The [anything] Alliance" gives it that hardcore "We'll form Voltron and smite you if you look at us wrong" street cred.

I truly wish them luck

[Bullfish]

I really do wish them luck. The thing is the "document" and "content" companies are going to fight like hell to expand proprietary formats as they ultimately look to the MS word format, the sheer number of copies of MSOffice sold, and see the dollar signs available by controlling the format and making everyone dance to their tune. Anyone who remembers the fiasco that occurred when MSOffice 97 wasn't very compatible with the previous version will also remember that companies simply shelled out for converters etc until MS issued a patch. They had no choice.

While packages like open office etc exist, they have for a while and are perceived as "not being ready for prime time" by most businesses. The only advantage many see is the ability to save as PDF (another proprietary format). For ODF to take hold, governments and some very large publishing concerns are going to have to adopt it. Else, not much will change and the march towards increasingly proprietary formats will continue.

Digital Dark Age My Ass

[brunes69]

The problem is not that there is no long-term storage. The problem is that we produce more useless data than ever before.

Really, who gives a f*ck about your 1.25 TB of crap? Or mine? We're just two ants in the anthill. You really think you can look up any substantial amount of information on someone who lived 200 years ago? Hell, try *50* years ago. Aside from public records like tax information and housing details, and maybe some family photos, you are likly to come up with bubkus, unless that person was famous.

It's going to be no different 200 years from now, and frankly I don't see the problem with that. Only in the past decade has everyone gotten this weird urge to try and archive and record every unimportant detail of their daily lives (see MySpace.com, blogging, etc). What they don't realize is no one really gives a crap today, and they sure as hell won't give a crap in 100 years.

Historians want to know about culture as a whole, not in bite-sized chunks. Aside from the major move-makers (politicians, *some* celebrities), historians won't be any more interested people's musings on shit like Paris Hilton than I am.

Re:Digital Dark Age My Ass

[Random_Goblin]

Really, who gives a f*ck about your 1.25 TB of crap? Or mine? We're just two ants in the anthill.

actually judging by what modern archeologists find really interesting, it is exactly what future archeologists will be interested in.

The little bits of detritus that make up individuals lives are far more interesting than the "big picture" history which is usually heavily loaded with spin, and therefore a bit of chore working out what actually was happening as opposed to what people wanted you to believe was happening.

the fact that people ARE musing on shit like Paris Hilton IS going to be interesting to future historians, because it gives an insight into how people were living their lives and what was important to them at the begining of the 21st Century.

all of those pictures from our camera phones, and whining live journals may not be a terribly flattering picture of our lives, but for an archeological point of view, it's exactly the sort of evidence you want.

Re:not that I would be against..

[16384]

Not anymore, now lyx support windows. See http://wiki.lyx.org/Windows/Windows

Canards

[Tony]

Any /.'ers have more info about Microsoft's format?

Get thee to Groklaw, my curious friend. The debate, along with fine technical details are found there.

On the other hand, the consortium (if you will) proposing a universal open document standard sounds more open and the proof will be in the implementation. Still, I'd like to know more specifically what that standard proposal is in detail.

The implementation is here. It's called "ODF," the "Open Document Format." It is the default file format of the Open Office suite of applications; KOffice is also moving (or *has* moved, I'm too lazy to look) to that format, as well. IBM's office suite will implement ODF.

Again, Groklaw has a lot of information, including pointers to the official specification.

Re:THE one truly open format?

[albalbo]

There are a few people pointing out other "open" file formats. OpenDocument is part of that heritage, though:

- it invents very little. The container format (jar) is well known, XML is well known, it builds on HTML for semantic structure, it uses other standards (XLink, XForms, SVG, etc.) where it makes sense to. It is, in effect, a "common subset" of standards which are all useful in creating documents (e.g., HTML is great, but you can't store images easily in an HTML file [tho obviously, yes, it is possible...]). This is in stark contrast to, e.g., MS XML.

- it has been well-designed from the start to actually improve the current state of the art, not replicate it. E.g., the metadata system is good and getting better.

- unlike text/html, competing implementations are actually interoperable: vendors are working through OASIS, which has standardised it from the start, and are making sure things work. HTML standards came a little late in the game, and there are still text/html pages out there which my poor Firefox can't handle.

There are a ton of reasons why OpenDocument isn't just "another format", but actually a significant step forward.

Dark Information

[GodWasAnAlien]

Historically, the problem of the disappearance "important" information has always existed, but some do not see the possible connection in a modern, digital world.

Some pieces of information did really exist long ago, but we only have references to the information, not the information itself. This could be from the lack of copies, or from suppression from religion or government.

In our digital world the same could happen with information, including software, books, music, and movies.

In an effort to absolutely control the information, different information industries attempt to control the media, using secrets, encryptions, and government control. These industries intend to profit from this information control as long as possible. The end of this control is assumed and mandated not to exist.

The problem is that at some point in the future the information could become non-valuable to these information industry. But currently, no mechanism exists such that these industries would be required or motivated to reveal the secrets or encryption mechanisms that would make the information useful. One cause could be that other information uses similar encryption or secrets, and the profit possibility of that information may be jeopardized.

The result is that unprofitable information may silently disappear, as whatever backups of the original expire.

Some examples would be:

A software company writes software, selling binaries only to the public. The copyright for the software is 100 years. Far before the end of the 100 years (perhaps 10 years),
the original source was no longer kept by the company. So in the future, looking back at the state of software in the year 2000, perhaps there may be some pictures of "Windows XP", but it may be unclear what it did, as no source exists, and it's not really worth reverse engineering. While somethings called Linux and BSD did exist, and the complete information/source about these would still be available. History can really focus only on the known, not the hidden.

Similarly, assume that the recording and music industry come up with the "perfect/unbreakable" encryption. They spend much of there resources hiding anything close to raw digital information from the consumer. But this DRMed songs eventually become unpopular. Obviously the DRM mechanism could still not be revealed as they still use it for other songs. They have essentially subverted any copyright limits, to impose an infinite limit. After the point of dis-interest, the DRM songs/movies may just fade away. I suppose Creative Commons music/movies of the time may survive instead. Obviously these may not represent what was seen at the time.

Re:not that I would be against..

[rxmd]

But there are already such formats. I.e. latex.

LaTeX is in no way an open document format suitable for storing, let alone archiving data, for a variety of reasons:

• Firstly, LaTeX gets its usefulness and power from packages. Unless you want to standardise on a given reference set of packages, it can't be used sensibly for archival purposes. because you'll have to store all possible packages in all versions along with your data. If you're willing to do that, you could run Word in an emulator, too.
  
• There is no universal method for package versioning, for resolving package dependencies and for maintaining backward and forward compatibility between package versions. This creates lots of problems when you use older documents on a newer TeX system. An example was the rather popular geometry package for easier page geometry setup where version 3 of the package broke compatibility with older versions. The author added a simple switch to make the new version behave like the old ones, but you had to add the switch to the \usepackage declaration to make your documents compile. If you have to modify your documents to keep them useable, you're missing the point of a document archive.
  
• There is no consistent way of using Unicode in TeX documents. Basically, with the existing solutions such as Lambda/Omega, UTF-8 inputenc, ucs.sty and proprietary packages, it's "choose two out of: compatibility with most LaTeX packages, compatibility with the Unicode standard, large character repertoire". It's somehow useable, but not really well enough to be called universal.
  
• LaTeX documents are really difficult to parse on a computer, making them even more ill-suited for archival storage on a large scale. Try talking to the developers of the TeX->LyX conversion scripts one day. Someone stated that the only good TeX parser is TeX itself. A good archival document format should be parseable using third-party tools.

LaTeX is a typesetting system. It's designed for getting a nicely formatted PDF or PostScript file out of a source file that you can alter and modify on the spot. Typesetting is what it does really well. If you try to shove and bend it into other roles, it starts to get kludgy, especially when it concerns data exchange between large numbers of users with inconsistent package versions, automated processing of LaTeX documents with third-party tools or heavy use of international character sets.