New York and Minnesota Publish Open Document Studies

Multiple readers have written to point out that New York and Minnesota have reached the end of their lengthy deliberations on open document formats. Both reports agree that an open format would be beneficial, but neither were willing to endorse a particular choice. New York's executive summary notes, "The State Legislature should not mandate in statute the use of any specific document creation and preservation technologies, as technologies can easily become outdated." Minnesota's report claims, "The marketplace is still in flux, and it is not certain that a single standard will emerge." In related news, yesterday's announcement from Microsoft that they would provide support for ODF in a future update to Office 2007 has EU antitrust investigators optimistic, but cautious. Microsoft has said that the ISO process was what prevented OOXML from receiving support in the same time frame.

Comments

Goes to show

[chuckymonkey]

Outdated? Really? Did they even bother to ask anyone that knows anything about opensource? Open standards means that even if it does become outdated there can still be an implementation no matter what. It's the best insurance against having outdated formats. I guess I'm off to write a few paper letters.

Re:Goes to show

[Anonymous Coward]

Open standards means that even if it does become outdated there can still be an implementation no matter what. It's the best insurance against having outdated formats.

Huge difference between restrictions related to a specific open standard and open standards in general. Creating a mandate for open standards in general in relation to government documents would be the best choice as opposed to any specific choice. If a standard is truely open then any document creating software can get the code written to read and output to the open standard. Newer but still open standards could be adopted under a wide mandate without having to change the mandate, but they should be provably open before adoption.

Then as you said it would be insured to be readable later, of course with one additional caveat, the documents should be stored on media with open source file structures as well, else that insurance could be blocked by the inability to read the old file structure. Old documents might be required to be moved to newer storage devices as well but as time passes on the process could become increasing expensive. How many former modes of storage hardware are no longer in use or at least no longer in wide spread use? However if such a requirement is made, then look for Microsoft and perhaps others to fight it as hard as they can.

Damn that ISO

[sc0ob5]

Yeah I'm sure it was the ISO process that prevented it from getting support, not the shoddy standard that is OOXML.

Re:Damn that ISO

[Anonymous Coward]

They are talking about what's preventing *Microsoft* from supporting it in Office 2007. The problem for MS is that OOXML as submitted by MS, and OOXML as approved by ISO is not completely the same. There are minor incompatibilites, which would prevent Office 2007 SP n-1 from fully understanding documents written with Office 2007 SP n. Microsoft doesn't want this situation, they only want to introduce incompatibilities with new versions. At least with a new version, customers have learned to expect incompatible documents. So Office 14 will create OOXML documents that Office 2007 won't fully understand. Office 2007 SP n will not.

Justification for inaction

[Excelcia]

Really, the use of fear of z standard becoming outdated is just a justification for inaction. There is no answer to that in any sort of technical field. What can you say when technology will always improve, standards will always become outdated. Saying you shouldn't adopt a standard because it will become outdated is precisely akin to saying you shouldn't drive a car because it will eventually run out of gas. It's just a mask to allow them to justify to the public why they won't move forward.

If fear of a standard becoming obsolete is a reason for not adopting it, I'm curious as to how they justify any of their IT budget?

Outdated laws are a problem

[l2718]

Legislation is difficult to change once passed. Competing interests (or lack of interests) and simple inertia mean that whatever gets written into law stays there for a while. It makes sense for the law to say that the standards should be open, that they should be chosen by a particular state agency, or that they should be reviewed every X years. But writing the choice of standards into law is very inflexible. There is a reason why we have building codes and highway codes. This doesn't say that administrative rulemaking is less subject to lobbying and corruption, or that it is more transparent. But on these counts it is no worse than primary legislation.

Re:Outdated laws are a problem

[chuckymonkey]

I agree with you that they shouldn't state in laws that a certain standard should be used, just that they be open. It was just their wording as to the reason that I don't particularly like. Anyway, when laws get passed stating that the standards for government documents must be open are there also penalties for non-compliance? These documents are important and the best way to ensure that they survive the test of time is to make the standards open and free to download by anyone.

Re:Outdated laws are a problem

[ozmanjusri]

I agree with you that they shouldn't state in laws that a certain standard should be used, just that they be open.

The common way to do it is to have legislation which refers to a standard managed by an external agency. The standard can then be changed without requiring legislative change.

This works well in fields like safety, where OHS laws can reference equipment like fall prevention harnesses, and still allow manufacturers the opportunity to innovate in their products.

It will fail in an arena where the resident monopolist is willing and able to trample standards bodies in order to perpetuate its monopoly. Until the monopolist is unseated, or demonstrably changes its ways, more specific legislation, such as mandating a particular format will be needed.

Clueless legislators...

[jkrise]

Both reports agree that an open format would be beneficial, but neither were willing to endorse a particular choice.
The State Legislature should not mandate in statute the use of any specific document creation and preservation technologies , as technologies can easily become outdated.

Looks like Microsoft has effectively brainwashed these clueless legislators.

Formats and technologies are completely different things. ODF is an Open Format. Open Office is a technological implementation of this format. Microsoft has recently merely proclaimed that they will also be implementing ODF in the current version of Office.

OOXML claims to be an open ISO certified format. But as on date, there is no technological, compliant iomplementation... in fact, the specification is not yet documented, as required by ISO processes.

Why can't they simply legislate on ODF, and then go about choosing the ideal technological implementation of the same?

Re:Clueless legislators...

[joelstobart]

I think what the legislators were saying was

• We want open-standards
• Every 2/4/6/8 years we'll meet and decide what those standards are to be reviewed by a technical team.

Other good news....

• iso OOXML is not going to be implemented by Microsoft until 2010! So no rational government can use that.
• (oasis) odf 1.1 will be supported by microsoft office 2007 in early 2009
• there are numerous other ODF choices.
• OOo3/Write/Google Docs are going to give office a run for its money

its a good year for document freedom.
- Joel

Re:Clueless legislators...

[ozmanjusri]

You do know there isn't a fully compliant ODF implementation either right?

More spin.

This statement is misleading. Every file written by OpenOffice.org, KOffice or IBM Symmphony (to use common examples) is ODF compliant. The file may not require every tag in the full specification to describe the contents each application is capable of writing, but it will comply with the standard.

In other words, each application is fully compliant with the subset of the standard mandated by the application's content creation role.

By contrast, MS Office does NOT write compliant OOXML files at all.

Re:Clueless legislators...

[oyenstikker]

Can OO.o, KOffice, or Symphony read _any_ valid ODF document properly and properly save any changes made to the document that ODF supports?

If not, they are correct in stating that there is no fully compliant ODF implementation.

Yes, outdated.

[marxmarv]

As in functionally outdated. What happens when active documents become the norm? (One could argue that with AJAX and magical PDF forms, they already are.) What happens when wikis are the expected way to receive complex information with cross-references? Another chance for content handling software houses to get their fingers sticky? In court, sometimes you don't appeal because you know you'll lose, and sometimes you don't appeal because you know winning would be worse than losing.

Composition of those states' panels?

[christian.einfeldt]

It appears that Microsoft is shifting its competitive strategy with regard to ODF from rejecting the ODF standard to fighting for control of the standard in the committees. Below is a statement from James Plamondon, Technical Evangelist, Microsoft Developer Relations Group, describing his view of how to leverage Microsoft's influence over committees. One wonders to what extend Microsoft might have used these tactics in New York and Minnesota:

have mentioned before the "stacked panel." Panel discussions naturally favor alliances of relatively weak partners -- our usual opposition. For example, an "unbiased" panel on OLE vs. OpenDoc would contain representatives of the backers of OLE (Microsoft) and the Backers of OpenDoc (Apple, IBM, Novell, WordPerfect, OMG, etc.). Thus, we find ourselves outnumbered in almost every "naturally occurring" panel debate.

A stacked panel, on the other hand, is like a stacked deck: it is packed with people who, on the face of things, should be neutral, but who are in fact strong supporters of our technology. The key to stacking a panel is being able to choose the moderator. Most conference organizers allow the moderator to select die panel, so if you can pick the moderator, you win. Since you can't expect representatives of our competitors to speak on your behalf, you have to get the moderator to agree to having only "independent ISVs" on the panel. No one from Microsoft or any other formal backer of the competing technologies would be allowed -just ISVs who have to use this stuff in the "real world." Sounds marvellously independent doesn't it? In feet, it allows us to stack the panel with ISVs that back our cause. Thus, the "independent" panel ends up telling the audience that our technology beats the others hands down. Get the press to cover this panel, and you've got a major win on your hands.

It bears scrutiny as to who was on the New York and Minnesota panels, and their affiliations and histories. I don't actually know of the composition of those panels, so I am posing a question, not making a statement.

No implementations of Microsoft's OOXML

[christian.einfeldt]

It is important to remember that there are no implementations in the real world of Microsoft's OOXML schema, as discussed in Andy Updegrove's blog, quoting David Worthington:

http://www.consortiuminfo.org/standardsblog/article.php?story=20080521092930864

Worthington's story includes quotes from Matusow and Mahugh that provide intriguing insights into how the decisions were made. After noting that saving to the OASIS ODF 1.1 format would now be possible, Worthington writes:

However, the company is not quick to embrace its own creation. Mahugh stated that Microsoft would not implement the final ISO version of OOXML until Office 14 ships at an unstated date in the future. This variant of OOXML was designated ISO/IEC 29500 at the time it was certified as an ISO International standard in April.

"One way to look at it is the prioritization of formats," Mahugh explained. "We reach a point in time where we have to decide whether to continue to invest in a previous version [of Office] or to cut the cord and move forward."

ODF support was a priority for Microsoft, Mahugh noted, adding that "real world" customers say that there is a pressing need for PDF [AU: ODF?] support. "At this point there are no products using [ISO/IEC 29500] in the marketplace."

When will Microsoft support its own file format? Worthingon quotes Gartner Research's Michael Silver on that question as follows:

"Customers that are expecting true document fidelity from XML-based, ISO-standard document formats will continue to be disappointed." Silver observed that the most compatible formats to use today are Microsoft's legacy binaries, and he believes that Microsoft will be unlikely to convince customers to move to OOXML in the foreseeable future.

Re:A little suspicious...

[a_n_d_e_r_s]

Microsoft has already said they will only implement ODF 1.1 when at the same time all competitors will produce ODF 1.2 documents.

Thus Microsoft will not be able to read many ODF dokuments produced because of imcompatibilities.

Its Microsoft making a imcompatible version of Java all over again to stop Java from being adopted. Now they are doing it to stop ODF from being adopted.

Embrace Extend Extinguish in 3..2..1..

[turing_m]

Minnesota's report claims, "The marketplace is still in flux, and it is not certain that a single standard will emerge."

Well, it looks like we standardized on MS-ODF. It has all the benefits of ODF, but with new, undocumented features that we all can't live without, such as uh, ribbontables.NET. We may not be able to read this format in 20 years by a non Microsoft application, but according to independent studies funded by Microsoft, our productivity will increase 150% in the meantime.

Executive summaries compared

[davide marney]

The report from NY has a more thorough treatment of the issues than the report from MN. It makes a strong case for openness as a policy, not a technical choice; shows how it is just one of many such choices, not all of which can be equally satisfied in every situation; and then very ably lays out a recommendation for making sure that openness gets pushed down into the state agencies.

The report from MN is focused on relating the wide variety of opinion that exists in this area, and not on making a specific recommendation (other than the commonsense one that the market is in flux and that the legislature shouldn't be picking market winners and losers.) The report gave me a much better understanding of just how confusing it can be when people try to talk about this issue. Like many complex topics, one needs to almost insist on agreements about terminology and scope even before engaging in the real discussion.

Re:Question: How does a format really get out of d

[Flambergius]

Question: How does a format really get out of date

There are several ways in which a format can become outdated. Below I will list some. You will notice that from a strictly technical point-of-view they aren't even close to being show stoppers (ie. you could work around them probably in several way). However, should that outdated format be mandated for use by a law then the technical challenges and financial burdens may become serious hindrances in the least and actual show stoppers in the worst case. Changing a law will always be more difficult than changing just a standard.

An ad hoc list of how a format can become outdated (pardon the poor examples):
  1) An underlaying technology or medium becomes outdated. Example: 8-, 16-, and 32-bit integers. Another example: pre-web/xml EDI-formats.
  2) A superior (= more fit) competing technology is developed. Example: SGML vs XML.
  3) The intended use case of the format becomes outdated and/or irrelevant, which may happen due multitude of technical and non-technical reasons (the world does not stand still). Example: an early text-processing format that does not support hyperlinks or embedded pictures. Data formats for various deprecated ports.

Your particular question was about "an XML format designed to represent a wordprocessed document. How exactly is that going to go out of date?" Let's first note that in practical terms this is a format specifically designed for longevity. However, it fairly easy to imagine that a word processing format designed today does not allow for all important future use cases. Information about intention might be very important in mere twenty years, as AI and cognitive modeling applications might require it. The format might lack important security features that become necessary way you interface with data via a brain implant.