Slashdot Mirror
Help Write An Open Data Format Bill
AdamBa writes "There has been a lot of discussion of open source bills, but I think open data format bills have a much greater chance of actually becoming law. Over at the Open Data Format Initiative site, I have written an article explaining "Why Open Data Format Laws Are Better Than Open Source Laws". I also have a sample Open Data Format bill; I invite comments from slashdot readers, in particular on how the sample bill could be improved."
Text of article, complete.
---BEGIN---
Why Open Data Format Laws Are Better Than Open Source Laws
With a variety of open source bills introduced, both in the United States and elsewhere, there has been a lot of discussion about open source laws. However open source laws have problems, both structurally and politically, and I think open data format laws would work much better.
(NOTE: I use the term "open source laws," although in fact some of the laws refer to "free software" instead.)
The reasons are as follows:
Open source laws are too easy to argue against
The three points mentioned most often in favor of open source laws are cost, security, and open data formats. In the lobbying against open source laws, I have never seen any negative comments about open data formats; the focus is on the cost and security arguments.
When discussing cost, opponents of open source laws can point out (correctly) that the actual cost of the product is only one part of the total cost; Microsoft quotes a Gartner Group survey putting the number at 8%. Presumably they found the study with the lowest number, but the general fact is correct. Plus, the cost issue likely favors open source more on the server, where administration costs may be lower with open source software; on the client, where Windows is bundled with almost any computer anyway and support involves helping end users with unfamiliar software, open source won't come out looking as good.
Now, you could argue that even the study that Microsoft is pushing shows that the total TCO of open source is only 92% of what it is for proprietary software. The problem is that this then leads to a long debate about how open source affects the other costs of software (installation, support, administration, etc) and no clear winner will emerge.
Meanwhile, the security issue can easily get embroiled in a FUD battle between the two sides, each claiming that the other has more crashes and remote exploits, each waving studies that support their claims. If you want to convince a legislature to pass a law causing significant, possibly risky changes in government procurement, you can't get stuck in a battle like that. Keep in mind that properly designed secure file formats are not dependent on keeping the file format itself secret, so nobody should be able to argue that open data formats compromise security.
When the debate can be framed in terms of cost and security, the issue of open data formats can be conveniently ignored by opponents. Requring only open data formats would remove the abililty of opponents to attack the cost and security arguments, leaving them to come up with arguments explaining why open data formats are bad, whch I have not seen so far. Finally, governments have presumably always considered cost as a factor when evaluating software purchases, and these days they no doubt consider security too; having a law that focussed only on open data formats would open their eyes to something new, that they have probably missed in the discussion of open source laws.
Open source laws are either too inflexible, or require too much work
Many open source laws seem designed to force a government to replace Windows/IIS/Office with Linux/Apache/*Office, but of course they aren't written that way; they discuss open source and proprietary software in general. This can take one of two tacks; either requiring open source or free software with no alternative, or making it difficult to buy proprietary software (for example requiring each purchase be accompanied by written justification).
The first approach takes too simplistic a view of the type of software that governments use. Much of the it is customized for specific tasks such as processing drivers' licenses, and the market for providers of such software is presumably small. If software vendors release their software as open source, they may find that cash-strapped governments in other states gladly help themselves to it for free, so the vendor may g
Who did what now?
I guess the best example would be Oracle and their proprietary database files.
To handle a case like this, the company would need to apply for an exception, which could be granted under the following conditions
a) an independant technical review confirms protecting the file format is significant to the business
b) the company provides free utilities to extarct all data into an open format from the proprietary format
c) The source code for the utility is escrowed, and publically released in the event of the company going under, discontinuing support of the product, etc
This would allow Oracle to protect it's way of storing a database, but doesn't prevent anyone from moving to a different database
I won't brattle on, but how about using a Wiki to fascilitate the creation of various public participation law draftings?
This would be an excellent application of the wiki technology and mantra of all-and-any user participation.
Just an idea...
cheers,
Levendis47
--==[ AOL YIM ICQ : Levendis47 : levendis47@yahoo.com ]==--
yes -- you were the only one.
And no, they are not really embracing XML -- read a little more about it. "Consumer Grade" (read: non-corp) will still use propriety format. The XML will only be on the high end, which should keep them nice and securely embedded in the consumer market.
I pushed for a similar "rule" at the local government offices I work at. All documents available to the public, and all documents used for communications with contractors/agencies/whatever, must be in an open format, unless no open format exists for the given data that conveys all needed information. (Which is pretty damn rare to not have.)
.DOC files from coworkers) but a good deal of external communication is on open formats. Our e-mail gateway blocks most non-open formats, which helps a good deal as well. ;-)
It's working fairly well. Enough of the employees are still just using the default MS formats in MS Office (I still receive enough
On a sadder note, tho, the residents have never requested this. Likely, they do not care; the majority of them, anyways. Increasing demand from residents would help push more gov't agencies to use rules similar to ours. How many of you Free Software and Open Communications geeks have even sent an e-mail to your local township/city/county/etc. requesting open formats? Truly, even a small handful of voices are listened to, from my own experience in the field.
Be heard!
...right but you lose information if you save an office document that way.
And even assuming that, as often as not the US government uses a proprietary data format exclusively. NSF now only accepts grant applications electronically. The forms are now an MS Office macro or an Adobe PDF form (which requires that you use the Adobe writer).
Why should I have to pay a third party to do business with the government? Especially when alternatives exist (or could if the government took interest in this issue) ...
You say
All these problems could be solved by simply having an "Export" feature, that spits out all the relevant data, relevant being defined as whatever the user has typed or otherwise entered. The major examples are going to be text documents, databases, and spreadsheets. This would also work for all the one-off homegrown apps that store whatever in some binary format.
Following this idea, the proposed bill could just list the benefits of open formats and mandate the ability to export to something like ASCII, RTF, CSV or XML.
What if life is just a side effect of some other process and God has no idea we exist?
This "open data format" proposal is impractical and counter-productive.
Impractical
Most computer programs are useful because they produce human-readable output, usually by arranging pixels on a screen or on a sheet of paper in ways that correspond with natural-language representations (words and numbers). Every purchaser of software is free to decide before purchase whether a program's output is sufficient for his needs, including any potential need for data archiving or transfer.
But until the data is output, it is encoded for the convenience of the program, not for the user. Even an ASCII file is not really "text," but numeric bytes whose values correspond to alphanumeric characters only through an artificial, though widely adopted, convention.
A data file is just a data structure stored in a persistent medium. As a developer, I shudder to imagine the burden of documenting every bit of data my programs store. I'd need to convert them from binary format to human readable. Not only that, but binary formats are necessary for some functionality. Have you ever benchmarked the performance of a large conventional database against a stored-as-XML database?
I guess I couldn't compress data, or maybe I could only compress it in a "standard" format - but which one, and who decides which one? Graphics and sound? I suppose I couldn't use a proprietary format for those either. And I guess I couldn't encrypt any data, at least not in any way a competing program couldn't decrypt. Well, if my competion can decrypt the data, so can anyone else. Sorry, but this is a non-starter for many business applications.
Counter-productive
Currently, a user has his software, he has his data (stored in whatever format the software's author found efficient and secure enough for the application), and he has a legal right to make copies of them so he won't lose any data he has bargained for. So this law doesn't really gain him data security; it gains interoperability, the possibility that competing software vendors will be able to serve his needs. There's an implicit hope that small software vendors could use open data formats to compete more equally against large vendors.
Alas, that hope is a pipe dream. Because any software vendor would be able to extend a published data format, the publisher who could extend it most and fastest would always lead the market. That means the biggest company would lead the market.
It's happened before. Microsoft's president is on record, telling his developers to "embrace and extend" industry standards. That's how Microsoft maintains de facto leadership of many standards. They can embrace whatever is open to be embraced, and extend it so it becomes Microsoft's. Examples include HTML (Microsoft's progress slowed by developer inertia), web services (Microsoft looking strong), Java (thwarted, so far, by Sun's license), and even ".doc" files (we old-timers remember when they were plain ASCII).
An open source author has the protections of copyright and licensing terms against Microsoft stealing his code. But the author of a program that keeps its data open has no such protection. The data belongs to the program's user, not the program's author. Open data is ripe for the picking by a bigger company, and a small-scale author of successful open-data software will soon be following instead of leading in his market.
Let's leave software authors and users free to decide whether human-readable output is enough for them, or if they want data structures stored persistently in human-readable form. The perversion of the patent system should teach us this much: laws that are meant to help the little guy, may help the big guy even more.
Dead people don't go to the emergency room, they don't collect unemployment or disability insurance, they don't get Medicaid or Medicare, and they never get any more social security benefits.
A seatbelt might save your life, but in anything but a non trivial crash you probably will still be injured. In such a case it probably saves resources to simply have the victims safely dead.
If someone doesn't want to wear a seatbelt (facing a greater possibility of death) then that is fine with me. Dead people cost me less money.
"Most computer programs are useful because they produce human-readable output, usually by arranging pixels on a screen or on a sheet of paper in ways that correspond with natural-language representations (words and numbers). Every purchaser of software is free to decide before purchase whether a program's output is sufficient for his needs, including any potential need for data archiving or transfer."
But the catch is that when the govt decides that the output is sufficient for its needs, publishes the output, and then in order to have access to it, the public needs to buy certain software. If the govt were required to publish data in an open format, that would solve this. They could do what they liked internally, but if the data was for public use, it would have to go into os format.
"I guess I couldn't compress data, or maybe I could only compress it in a "standard" format - but which one, and who decides which one? Graphics and sound? I suppose I couldn't use a proprietary format for those either. And I guess I couldn't encrypt any data, at least not in any way a competing program couldn't decrypt. Well, if my competion can decrypt the data, so can anyone else. Sorry, but this is a non-starter for many business applications."
This is just pants, it's not the the crypting algo that's secret, it's the key.
I am the author of the Oregon Open Source bill.
You are seeing things exactly as I saw them when I wrote the original bill: I felt that its real power was not in the open source provisions -- those were there to get media attention -- but in the requirements for open standards.
I was unable to contact the one person I needed help from when writing the bill -- Larry Rosen of the Open Source Initiative -- until after the bill had been introduced with all of the flaws and mistakes I made.
Please, get advice from Larry Rosen in writing your sample bill. I won't post his contact info here, but I'm sure you can find him if you look.
Ken BarberIn times of universal deceit, telling the truth gets you modded -1 Troll
Microsoft was also one of the main companies lobbying against open source bills, thus it will presumably be one of the main companies lobbying against open data format bills...so it makes sense to think "how can this be written to deflect arguments Microsoft will make".
- adam
This open formats idea is naive. You don't cut deals with monopolies, you cut their goddam welfare. Do you realize how much every single elementary school in your state if forking over to these whores? The DMV is a lame example. The schools are where they're juicing the taxpayer like crazy. If your kids have a web page at school, I suggest you go see which server they're using. Oh, hmm. Who made that decision. Oh I see, that was the district information guy. Where is he anyway? Oh, he's at a Microsoft conference in Vegas. There's so much to learn you know.
Where do you think Mac got all that money that keeps them afloat? They got so bloated off edu-welfare they went limp, pulled out and gave MS a turn to ride the train. How the hell is welfare bad for the poor, but totally justified in the case of billion dollar corporation? I still don't catch that part.
Fuck the half steppin'. Get out and vote for mandatory open source and do the right thing. Don't be a suck ass apologist with this "The right tool for the right job" bullshit.