Slashdot Mirror
Help Write An Open Data Format Bill
AdamBa writes "There has been a lot of discussion of open source bills, but I think open data format bills have a much greater chance of actually becoming law. Over at the Open Data Format Initiative site, I have written an article explaining "Why Open Data Format Laws Are Better Than Open Source Laws". I also have a sample Open Data Format bill; I invite comments from slashdot readers, in particular on how the sample bill could be improved."
Now, you could argue that even the study that Microsoft is pushing...
Although proponents of these laws may dream that Microsoft is going to open up their source...
If the government uses Office and the Office data format is made public...
Either Microsoft and other vendors are going to have to completely change their business model...
Microsoft will undoubtedly still lobby against open data format laws, but its arguments will be weakened significantly...
It doesn't have to get into reasons why open source laws would be bad for Microsoft personally...
Are there any companies you think these laws will effect? Oh wait....
Vonal Declosion
Text of article, complete.
---BEGIN---
Why Open Data Format Laws Are Better Than Open Source Laws
With a variety of open source bills introduced, both in the United States and elsewhere, there has been a lot of discussion about open source laws. However open source laws have problems, both structurally and politically, and I think open data format laws would work much better.
(NOTE: I use the term "open source laws," although in fact some of the laws refer to "free software" instead.)
The reasons are as follows:
Open source laws are too easy to argue against
The three points mentioned most often in favor of open source laws are cost, security, and open data formats. In the lobbying against open source laws, I have never seen any negative comments about open data formats; the focus is on the cost and security arguments.
When discussing cost, opponents of open source laws can point out (correctly) that the actual cost of the product is only one part of the total cost; Microsoft quotes a Gartner Group survey putting the number at 8%. Presumably they found the study with the lowest number, but the general fact is correct. Plus, the cost issue likely favors open source more on the server, where administration costs may be lower with open source software; on the client, where Windows is bundled with almost any computer anyway and support involves helping end users with unfamiliar software, open source won't come out looking as good.
Now, you could argue that even the study that Microsoft is pushing shows that the total TCO of open source is only 92% of what it is for proprietary software. The problem is that this then leads to a long debate about how open source affects the other costs of software (installation, support, administration, etc) and no clear winner will emerge.
Meanwhile, the security issue can easily get embroiled in a FUD battle between the two sides, each claiming that the other has more crashes and remote exploits, each waving studies that support their claims. If you want to convince a legislature to pass a law causing significant, possibly risky changes in government procurement, you can't get stuck in a battle like that. Keep in mind that properly designed secure file formats are not dependent on keeping the file format itself secret, so nobody should be able to argue that open data formats compromise security.
When the debate can be framed in terms of cost and security, the issue of open data formats can be conveniently ignored by opponents. Requring only open data formats would remove the abililty of opponents to attack the cost and security arguments, leaving them to come up with arguments explaining why open data formats are bad, whch I have not seen so far. Finally, governments have presumably always considered cost as a factor when evaluating software purchases, and these days they no doubt consider security too; having a law that focussed only on open data formats would open their eyes to something new, that they have probably missed in the discussion of open source laws.
Open source laws are either too inflexible, or require too much work
Many open source laws seem designed to force a government to replace Windows/IIS/Office with Linux/Apache/*Office, but of course they aren't written that way; they discuss open source and proprietary software in general. This can take one of two tacks; either requiring open source or free software with no alternative, or making it difficult to buy proprietary software (for example requiring each purchase be accompanied by written justification).
The first approach takes too simplistic a view of the type of software that governments use. Much of the it is customized for specific tasks such as processing drivers' licenses, and the market for providers of such software is presumably small. If software vendors release their software as open source, they may find that cash-strapped governments in other states gladly help themselves to it for free, so the vendor may g
Who did what now?
This idea seems too obvious, too clear, too intuitive, and far too easy to implement for any respectable lawmaker to consider it for even a single nanosecond.
To ensure perfect aim, shoot first and call whatever you hit the target
...might or might not now help linux ... a lot of people I've talked to just don't like openoffice (and I've noticed that big spreadsheets are intolerably slow). There's a bit more work to be done there, besides the standard ms office problems. (That said, it wouldn't hurt a bit). But it would definately help a company like apple.
Imagine flawless powerpoint support in keynote.
You say
Was I the only one who saw the title Help Write An Open Data Format Bill and thought "Yeah right, like Bill's ever gonna do that."
Rock on M$
Yes, I know they are embracing XML now...
Hmmm...
The author has the right idea; it would certainly make building compatible third-party implementations of data-processing software (by that I mean nearly all software) much easier than if the file formats were closed, as they are now.
The trouble is that the distinction between code and data isn't as bright a line as you might think. I don't mean code/data segmenting; if you think about what evals and ELF are, you know what I mean.
Algorithms (not to mention software in the US) are certainly patentable -- and efficient data storage mechanisms are too. Think about what a gold mine in terms of IP value a hash table would have been if it invented at a commercial organization instead of in academia.
So, the conundrum we still face is that there are still valid IP claims WRT data structures, because oftentimes, as much thought is put into them as into the processing software that reads and writes them.
On the second bullet you say "computer data owned by the [government] be permanently available to the [government] throughout its useful life."
Somewhere you may want to define "useful life", I could see this as a possible loophole, this term could mean different time lengths to different people.
Stuff this open data format crap.
I think data formats are patentable... because some of them are inventive and require real ingenuity. eg: video streaming, etc.
If you don't award patents, you don't get the great inventions. (cf: Edison)
from,
your friendly patent attorney
Goverment? What goverment? We are talking about open data formats for Internet, not for national network of one country, right? If so then the law should be international. Otherwise - it's wasting of time and efforts.
Less is more !
"A language that doesn't affect the way you think about programming, is not worth knowing" - Alan Perlis
I guess the best example would be Oracle and their proprietary database files.
To handle a case like this, the company would need to apply for an exception, which could be granted under the following conditions
a) an independant technical review confirms protecting the file format is significant to the business
b) the company provides free utilities to extarct all data into an open format from the proprietary format
c) The source code for the utility is escrowed, and publically released in the event of the company going under, discontinuing support of the product, etc
This would allow Oracle to protect it's way of storing a database, but doesn't prevent anyone from moving to a different database
I won't brattle on, but how about using a Wiki to fascilitate the creation of various public participation law draftings?
This would be an excellent application of the wiki technology and mantra of all-and-any user participation.
Just an idea...
cheers,
Levendis47
--==[ AOL YIM ICQ : Levendis47 : levendis47@yahoo.com ]==--
I would prefer a complete free and open standard with no patent encumberment, but a second-best option would be to have an open and fully documented standard that may or may not be subject to patents, along the lines of the stuff put out by the MPEG consortium (MP3, AAC, etc. are all patented, but are also completely open formats). This way at the very least you have an guarantee that your data won't be permanently inaccessible, because when the patent expires you can reimplement the open specification.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
a) Lobbyists from companies that currently have govt. contracts will be pissed
..NASA, FDA etc.)
b) cost to convert and/or interoperate with legacy systems will be IMMENSE, may run into the many billions (consider all the govt. agencies
c) Did I mention the costs involved (perceived costs, not necessarily real costs)
Nice dream though, but I rather dream of sleeping.
Imagine flawless powerpoint presentations in /dev/null.
Banaaaana!
Oregon's HB 2892 has sections covering both open source and open formats. It has recently been revived, and there's still a chance of passage.
I pushed for a similar "rule" at the local government offices I work at. All documents available to the public, and all documents used for communications with contractors/agencies/whatever, must be in an open format, unless no open format exists for the given data that conveys all needed information. (Which is pretty damn rare to not have.)
.DOC files from coworkers) but a good deal of external communication is on open formats. Our e-mail gateway blocks most non-open formats, which helps a good deal as well. ;-)
It's working fairly well. Enough of the employees are still just using the default MS formats in MS Office (I still receive enough
On a sadder note, tho, the residents have never requested this. Likely, they do not care; the majority of them, anyways. Increasing demand from residents would help push more gov't agencies to use rules similar to ours. How many of you Free Software and Open Communications geeks have even sent an e-mail to your local township/city/county/etc. requesting open formats? Truly, even a small handful of voices are listened to, from my own experience in the field.
Be heard!
Except the author thinks that software companies are just going to lovingly hand over specifications on their file formats. Throughout computing history, software companies have used file formats to lock people into their product, force them to buy upgrades, and use them as leverage for strategic partnerships. Ie- it's a huge cash cow to have proprietary formats.
The author, in the very first paragraph, dismisses "open source laws", saying they "won't work". Huh? Says who? Then he says his WILL work. You can't just make enormous blanket statements like that without backing them up!
In his second point, he says "Many open source laws seem designed to force a government to replace Windows/IIS/Office with Linux/Apache/*Office". Where is he getting this crap from? He goes on to dismiss the security benefits of open-source software, the cost savings...he pretty much dismisses every single argument for considering open-source software, with nothing to back up his reasoning. He 'thinks', therefore it is.
My impression of "open source laws" was that they instruct government agencies to -CONSIDER- open source solutions- it does not FORCE them to use them- yet he makes it out like there's a cop sitting there with a gun to a government IT manager's head, saying "Go ahead punk. Make my day. Install Office. Are you feelin' lucky, punk?" Maybe I misread it, but his idea seems to be to -FORCE- companies to release file formats if their software is used by the government.
One requires you consider open-source software, leaving commercial software companies plenty of oppertunity to compete if they've got a better solution(remember, open-source is not $0, you still have labor involved, possibly a migration, maybe staff training and hiring, maybe different equipment.) Open source doesn't automatically, if ever, "win" outright simply because it's open-source, yet the author seems to think open-source always will, and hence open-source laws will be bad because there will be a huge inconvenience to the government or the software companies. Again, consider- not force! If the manager thinks commercial software will overall be better, he/she will make that call.
The other forces software companies to do something that threatens, in a BIG way, how they compete against other companies.
Now, which has more of a chance of failing?
Please help metamoderate.
Admittedly i read the 'bill' a bit quickly, but there is one problem (which seems to be partially addressed later on) is that all data be in human readable format. Well this is good for string data. However I very much suspect there are going to be number formats that are much better stored in their binary format. Now as I recall from college COBOL's strategy as to have everything stored as string data with (interesting) defining headers. As I also recall this was to help portable between COBOL and other foreign (computer wise) data retrieval systems, like when data was passed around on tapes.
This is all well and good, but you trade off portability for parsing overhead. Of course CPU time is way cheap as to compared to yesteryear. However you will end up with people's nutjob parsing routines, I mean hell CPUs and calculator software still get shipped making math mistakes. Reading in data in any of the standard int and float storage formats is cheaper and you don't have to worry about joe-bob desk jockey hosing up an import routing and your tax bill being 100x too large.
Now as most of us who have been thru a proper programming class know there are routines for these kinda things.. but again beware of joe-bob who wants to do it 'better' or whatever. Perhaps along with open data one needs to define a data import stadard?
Remember DOC, DOC, DOC, DOC, DOC, DOC, DOC, DOC, ....
Open data formats are a good start, but don't fool yourself into believing they are sufficient. HTML is perfectly open, but how many web sites are there that test against anything but IE? How many rely on the behavior (even the bugs) of IE?
The non-openness of Microsoft Word's document formats is not the biggest obstacle. There are plenty of office suites now that can read them. The biggest obstacle is that people write their documents expecting them to look exactly as they do in Word. That's not something that open data formats by themselves will solve. Microsoft could move entirely to XML, and that problem would still remain.
This is like saying ASCII works for everyone. It is more than just the format on which the "real format" is based on. It would be ideal to have a full specification or detailed, non obfuscated documentation.
S
At the session, I got the impression that most government agencies are moving towards PDF for records than need electronic retention.
Not really. I used ClarisWorks when I was a kid, then I moved to StarOffice and then OpenOffice.
:-)
I never by choice used MS Office tools
Tom
Someday, I'll have a real sig.
Instead of making major corporations, such as Microsoft, open their formats, perhaps Word and similar programs should be required to have the ability to read and/or write major open formats, such as StarOffice/OpenOffice formats.
Yes, I could see arguments along the lines of "but we can't possibly support every format," but in the case of behemoth corporations, it should be doable for at least the major formats. This could also be said to stifle competition, but in Microsoft's case, it has already done enough of this, and this measure could actually restore competition.
I've read Grocklaw. BoycottNovell, you're no Grocklaw
This "open data format" proposal is impractical and counter-productive.
Impractical
Most computer programs are useful because they produce human-readable output, usually by arranging pixels on a screen or on a sheet of paper in ways that correspond with natural-language representations (words and numbers). Every purchaser of software is free to decide before purchase whether a program's output is sufficient for his needs, including any potential need for data archiving or transfer.
But until the data is output, it is encoded for the convenience of the program, not for the user. Even an ASCII file is not really "text," but numeric bytes whose values correspond to alphanumeric characters only through an artificial, though widely adopted, convention.
A data file is just a data structure stored in a persistent medium. As a developer, I shudder to imagine the burden of documenting every bit of data my programs store. I'd need to convert them from binary format to human readable. Not only that, but binary formats are necessary for some functionality. Have you ever benchmarked the performance of a large conventional database against a stored-as-XML database?
I guess I couldn't compress data, or maybe I could only compress it in a "standard" format - but which one, and who decides which one? Graphics and sound? I suppose I couldn't use a proprietary format for those either. And I guess I couldn't encrypt any data, at least not in any way a competing program couldn't decrypt. Well, if my competion can decrypt the data, so can anyone else. Sorry, but this is a non-starter for many business applications.
Counter-productive
Currently, a user has his software, he has his data (stored in whatever format the software's author found efficient and secure enough for the application), and he has a legal right to make copies of them so he won't lose any data he has bargained for. So this law doesn't really gain him data security; it gains interoperability, the possibility that competing software vendors will be able to serve his needs. There's an implicit hope that small software vendors could use open data formats to compete more equally against large vendors.
Alas, that hope is a pipe dream. Because any software vendor would be able to extend a published data format, the publisher who could extend it most and fastest would always lead the market. That means the biggest company would lead the market.
It's happened before. Microsoft's president is on record, telling his developers to "embrace and extend" industry standards. That's how Microsoft maintains de facto leadership of many standards. They can embrace whatever is open to be embraced, and extend it so it becomes Microsoft's. Examples include HTML (Microsoft's progress slowed by developer inertia), web services (Microsoft looking strong), Java (thwarted, so far, by Sun's license), and even ".doc" files (we old-timers remember when they were plain ASCII).
An open source author has the protections of copyright and licensing terms against Microsoft stealing his code. But the author of a program that keeps its data open has no such protection. The data belongs to the program's user, not the program's author. Open data is ripe for the picking by a bigger company, and a small-scale author of successful open-data software will soon be following instead of leading in his market.
Let's leave software authors and users free to decide whether human-readable output is enough for them, or if they want data structures stored persistently in human-readable form. The perversion of the patent system should teach us this much: laws that are meant to help the little guy, may help the big guy even more.
No, it was about choice. He was saying you should not be forced to do something that is common sense; that's a debate about choice.
I'm going to ignore most of your post on the grounds that you either have deliberately missed my point, used your own strawman arguments, or are just thick (I'll let you decide which...). However, there's a key point I want to address:
So you're saying the government is allowed to (pre-)determine for me a choice that could affect my living or dying? Do you think they should be able to do this in every case? What if I am going to have surgery where I have a 10% chance that I could die or be severely maimed? Because it's "only a 10% chance" does that mean the government gets to decide for me?
Actually, in general, yes I do believe that.
Most modern societies believe that we have a duty of care to other members of our societies. This results in such things as publicly funded hospitals.
Such facilities are almost always heavily utilised. In such situations, most societies look at ways of reducing the need for these services.
The seatbelt case is an excellent example. When seatbelts first started being available in cars, some bright spark noticed that there were relatively fewer injuries amongst seatbelt wearers. Statistical studies were done and they showed that, on the whole, wearing a seatbelt meant you were less likely to be hurt. Local governments started passing seatbelt laws. Other studies were done showing a large correlation between seatbelt laws and reduced hospital demand. So everybody picked it up.
If seatbelts weren't required, hospitals would have even greater demands on them. Hence the law. I feel that governments have the perfect right to curtail behaviour that puts higher demand on government services.
So, that's how a private choice (the decision to wear or not wear a seatbelt) becomes something worth legislating.
This is the dumbest argument. Those people who die or are severly hurt by seat belts should not have been legally bound to wear them. Perhaps they would've worn them anyway, perhaps not, but at least it would've been their choice and not the government's.
First obvious point: It's always their choice. The car doesn't have a sensor in it to not let it start if the seatbelt isn't worn. They can drive without a seatbelt. It's just now, in addition to risking greater injury in the event of an accident, they also risk getting fined.
Second obvious point: Yes, seatbelts sometimes result in greater harm than they do good. Laws are aimed at the benefit of society as a whole, not individuals. On the whole, society is helped by seatbelt laws.
To take another example, some people think vaccinations should be a personal choice. Vaccinations occasionally have undesirable side effects. Despite that, high vaccination rates for measles, polio, chickenpox, whooping cough, dyptheria, and other nasty diseases have vastly reduced breakouts and associated fatalities. However, it's also been shown that when vaccination rates drop below a threshold (like, 80%), the breakouts start occuring again. Because of this, if a government wants to make vaccinations mandatory, then I think they have every right to.
The desire to have governments not intrude on these apparently individual (and often common sense) choices is, at heart, an entirely selfish one. It ignores the fact that these laws are not designed to curtail your rights, such as they are; they are designed to help society as a whole.
And to tie it back to the topic: a government body mandating that suppliers of software to it use a publicised data format is a good thing. It may be common sense ("Of course I want the ability to access my data in any way I want"), but that doesn't mean it's not something that should be legislated.
"Software is too expensive to build cheaply"
Dead people don't go to the emergency room, they don't collect unemployment or disability insurance, they don't get Medicaid or Medicare, and they never get any more social security benefits.
A seatbelt might save your life, but in anything but a non trivial crash you probably will still be injured. In such a case it probably saves resources to simply have the victims safely dead.
If someone doesn't want to wear a seatbelt (facing a greater possibility of death) then that is fine with me. Dead people cost me less money.
The first approach takes too simplistic a view of the type of software that governments use. Much of the it is customized for specific tasks such as processing drivers' licenses, and the market for providers of such software is presumably small. If software vendors release their software as open source, they may find that cash-strapped governments in other states gladly help themselves to it for free, so the vendor may get only one paying contract instead of fifty. Therefore, it's quite possible that governments won't be able to find companies willing to provide them with open source software, and then what alternative do they have?
I just covered this recently in an op-ed piece in the business journal. Let me make this simple: governments do not exist to provide business opportunities to software developers. Reread that until it makes sense.
We (collectively "the people") shouldn't have to pay twice for a piece of software. The open source world needs to come up with solutions to boring problems, like drivers' license registration. If a particular company doesn't want to be a part of that, then so be it.
Michael
Do you have ESP?
The problem in your argument is that, despite what the ads say, seatbelts don't save lives.
Yes, that's right. Your chances of death in a car accident (at least, death without getting to a hospital first) are not decreased significantly by wearing a seatbelt. Seatbelts prevent the "being tossed around the car, suffering massive bruising, broken bones, spinal damage and internal bleeding" type of injuries. Most quick deaths from car accidents occur due to broken necks from whiplash (seat belts don't help, may hinder), or foreign object intrusion.
So how are your feelings of someone risking greater, and more costly, injury, but without raising their chances of cheaply dying very much?
(Nice pragmatic attitude, BTW... reflects mine a lot)
"Software is too expensive to build cheaply"
If software vendors release their software as open source, they may find that cash-strapped governments in other states gladly help themselves to it for free, so the vendor may get only one paying contract instead of fifty.
IIRC, the GPL specifies that you only need to make the source available to the entities to which you distribute binaries. I.e., if Michigan gets a contract with some company for some software, the company can release the software as GPL and only provide the source to the Michigan state government. It is then in the best interests of the state government to NOT redistribute the software. That way another state, say New York, also buys the software to make sure that the company is still there next year. The best thing is that if company X pisses off the state, they can take the source and hire company Y to maintain it. If I understand the intent of the GPL, this is the kind of choice that is available with free software.
Compare that to a previous post (I don't recall the exact thread) where the poster pointed out that his organization used a particalur personnel management product. PeopleSoft bought the competitor and disconitnued the product. Security flaws were found, his company had to spend $2M to switch to PeopleSoft. Had that software been GPL, they could have just hired another company to fix/maintain it. Again, in that situation it is in the company's best interest to NOT freely (as in beer) redistribute the program.
I think the same can be said for many non-commodity software products. I mean, how many people actually need or want (or have the hardware) to run a PeopleSoft level program at home? Better yet, if a company pays, say $250000, for a program and accompanying support, what incentive do they have to turn around and give it away? It seems to me that the GPL is really perfect for situations like this.
I am the author of the Oregon Open Source bill.
You are seeing things exactly as I saw them when I wrote the original bill: I felt that its real power was not in the open source provisions -- those were there to get media attention -- but in the requirements for open standards.
I was unable to contact the one person I needed help from when writing the bill -- Larry Rosen of the Open Source Initiative -- until after the bill had been introduced with all of the flaws and mistakes I made.
Please, get advice from Larry Rosen in writing your sample bill. I won't post his contact info here, but I'm sure you can find him if you look.
Ken BarberIn times of universal deceit, telling the truth gets you modded -1 Troll
Specifically, the applications must be able to save by default in the archival quality format.
This brings to mind the discussions of technological obsolesence that surfaced briefly in computer magazines a year or three ago. It's a timely subject, even if it is forbidden by Chairman Bill.
Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
Oregon has been cited for introducing open source legislation: but unfortunately is also one of the few states that have allowed state and local government entitites to charge development fees and a marginal copying fees for certain types of information (Oregon Revised Statues, section 190).
The federal government passed the 1995 Freedom of Digital Information, but not many states have followed similarly with their state government data... thus some states are are charging for data that isn't or can't be produced in the private sector (aka lucrative government data monoplies).
Access to government data *period* should also be a considered in the open source/format debate. Don't just assume that government data is available, it isn't in instances where it should.
I've seen all the comments on loopholes and inconsistencies in this proposed law. This is why the politicians and corporations normally write the laws.
Unlike us, they know how to eliminate loopholes and gross internal inconsistencies. Take, respectively, the tax laws and the DMCA for example.
If you have a law forcing data to be in properly-documented formats, proprietary software companies are free to implement apps that work with that format, without open-sourcing anything.
.doc format works".
They'll just have to compete (with open-source apps, and with other proprietary apps) on features, ease of use, reliability and things like that, that actually help the user, as opposed to competing on "but we're the only people who *really* know how
(IMO governments should use open formats; private companies shouldn't be required to use open formats, except when they interact with governments, which would hopefully mean they see the sense of going with open formats throughout.)
(Of course, if someone decides the best way to document their format is to open-source a sample implementation, that'd be nice too...)