Slashdot Mirror
Princeton Researchers Say Feds Need Data Standard
dcblogs writes "The federal government's data-sharing efforts are a mess, and if Barack Obama really wants a useful 'Google for government,' he would have to set the government's vast amount of data free by exposing it and ensuring it complies to standards. Once that happens, commercial sites, aggregators, bloggers and everyone else will be able to access it, use it and transform it, argue a group of Princeton researchers (follow Download link for full PDF)."
I'm not certain I would agree with setting it all free. However it all does need to be standardized.
As for the rest. WEB 2.0!
Barack Obama really wants a useful 'Google for government,'
Well, so far the government has bought parts of Bear Stearns and AIG. Maybe it's time they diversify into some technology companies like Google? Hell, let's buy them too!
I'm a big tall mofo.
I hear that Microsoft is already working on the problem with their proposed "Government Open XML" standard. This should not be confused with GOXMLb ("Google Open XML beta") because Microsoft would never try to confuse people on such an issue.
It is going up for ISO vote next week. Be there*.
(*) it will be very profitable for you to "be there".... nudge, nudge... wink, wink...
What you need is not one [set of] standard(s) but one vendor controlling and maintaining those standards... they know what is best for all of us because they are paid professionals, not hack hobbyists.
(Yes! I am kidding!!)
Seriously, it took a team of researchers to figure this out?
Palm trees and 8
One thought has occurred to me as part of this "sharing". Privacy and the other is Security.
Shai Schticks:"You don't make peace with friends, you make peace with enemies"
The government could hire librarians to organize the data. This is are a group of people highly trained in how to take large quantities of non standard data and organize it in a way that people can find what they want.
The government in the recent past standardized the way grants are applied for. The NSF, DOE, NIH all had different ways of doing things, but now it's through one process. Strangely enough, once they standardized everything, we went from 1 accountant who spent about 1/4 of his time fixing and processing prof's grant apps, to almost 3 full time people.
When a standard encompasses too many domains and departments, it just gets too large to be useful.
Remember the good old days, when transparency in government could be safely considered a good thing?
Generally, I'm still for it. Absolutely we need transparency in our government, and anything that brings us closer to point-and-click convenience over what we have now (FOIA requests left behind the radiator for 9–18 months to age and mellow) is for the best.
Furthermore, an open, accessible standard (i.e. no copyrighted DTDs, and I'm looking at you, Microsoft) will allow government resources to be brought together in interesting and inspiring ways. You know all those Facebook apps and Google Maps mashups? Imagine those applied to governance. The idea behind them is to put information together in new and interesting ways. If not only those in government, but the citizenry, can create government hacks like that, there would be great benefit.
Now let's talk hazards.
When was the last time you published your name and address online? See any good uses of microformats on any major sites lately? That's because there are some people on the Internet who are <sarcasm class="churchlady">not so nice</sarcasm>, and might willingly abuse whatever information they can find. The "government hack" alluded to above is an invitation to abuse. And we really can't afford to put government in that kind of position.
Another consideration, and I've stated this before, is that a wide line must be maintained between security and transparency. Security means that everything that must be kept secret is really kept secret. Transparency means that everything that doesn't have to be secure is made available somehow. If things aren't secured, the government becomes ineffectual and even detrimental. If things aren't kept transparent, the government itself can become abusive. A freely searchable infrastructure would make the transparency all that much more powerful, and make any breaches in security that much more severe.
You cannot truly appreciate Dilbert until you read it in the original Klingon.
I use lots of government supplied data in my work, and one constant has always been that the more work the agency does to make the data easily available, the harder the data are to use. Spreadsheets get posted with labels and data mixed, because that looks better in print. Spreadsheets get posted as PDFs, because that looks better in print. Footnotes and other textual material is mixed into numeric fields, because that is the way the material will be published in hardcopy. etc etc etc.
Databases get posted to the web with "interfaces" that allow single rows to be downloaded, but require months of screen scraping to get the entire database. Databases get released with (windows-only, of course) software with the same effect. etc etc etc
The reason is mostly that agencies want to discourage outside analysis of the data - they would prefer to avoid inconsistent messages getting to OMB or congress.
have already created information sharing data model standards for law enforcement and justice purposes. These include NIEM (National Information Exchange Model) and GJXDM (Global Justice XML Data Model). If the government can create these then additional models can be created for sharing information with its citizens. Someone (or group) just has to take the lead to do so.
this nation, under God, shall have a new birth of freedom. -- Lincoln, Gettysburg Address
http://www.fgdc.gov/standards
putting the 'B' in LGBTQ+
Come on, a bunch of Princeton researchers, after spending $X millions of grants from the US gov't over 5 years now says we need a data standard?
Dublin Core?
FEA? (Federal Enterprise Architecture, the other DRM)
OAIS?
And talk to anyone in gov't IT today on fed data problems, and they'll give you better info on how to solve the data issues vs. these researchers. Note to the Princeton researchers: stick to solving the semantic web problems--cause that's something we can ignore for the next 10 yrs.
And Google for Gov't may not be a good thing--google tracks everything. Now you know why gov't wants the same system.
ensuring it complies to standards
Hopefully he knows the meaning of a TRUE standard, as opposed to the other kind!
"The federal government's data-sharing efforts are a mess" This is very well known and real PRs would not need to state the obvious.
We The People really wants a useful government, but continue to elect tent-revivalist and pick-pocket politicians (most not all are in on the $7B scam).
Set the government's vast amount of data free is total bullshit.
Globally exposing Gov/Mil data/content IS NOT REQUIRED to ensure web-services/SOA and data/content complies with "Open" standards like UTF, ODT, PNG/SVG..., syntax/XML..., semantics/OWL..., synergy/collaboration, topology, ontology... [START HERE: en.wikipedia.org/wiki/OpenDocument_technical_specification, then don't stop RTFM... continue ISO, W3C, OASIS... google some of the words/concepts above... CONTINUE].
"Open" (as in 100% non-Proprietary) ISO/OASIS/W3C... hardware, software, and services International Standards (not corporate/MS/ATT/IBM...) can meet all NetCentric, GIG, ISE requirements without exposing data/content.
Once "Open" Standards happens, corporations will lose a large part of the strangle hold on US/EU Government operations, contracts, and acquisitions. Gov-transformation will happen, but for now much of the Defense Industry (EU and US) act more like the enemy than reliable patriotic allies for US & EU.
"A group of Princeton researchers (follow Download link for full PDF)" is either (or all) not understood by DCBlogs, an early 20090401 prank, or corporate BS attempts to keep the Government (technology clueless C*Os/managers) consuming bullshit from (technology clueless) Biz-Buzz marketeers.
Unaccountable leaders are masters, and unrepresented people are slaves. How do US and EU fare?
Government agencies do a lot of good stuff. A few bad apples pass contracts only to friends.
Effect: Agencies have to publicly request for bids
No we know if a three man team is not capable of building a power plant, so there might be more to consider than simply the price of the bid.
This is where standards came in: In order to lock the small companies out, the government requires bidders to adhere to ridiculous standards like ISO9000 or CMMI Maturity Level shit whose overhead only giant companies can afford, since others are busy working. The inventors get their ass stuffed full of money for certifying fees and another big chunk of tax dollars goes into useless forms that are required by a "mature process".
Now some researchers claim, that everything will be better if we make every agency adhere to standards. So now everyone will get the information he needs because some office uses xml? I don't think so! This will only force software onto government agencies that they don't want -- often enough for good reasons, and force more companies into the perverted process that is standardizing, which is again a big companies' game.
Standards came into being because people wanted to agree on something, which would be useful for everybody. If a standard was passed that was unneeded it was simply ignored. Making something a standard doesn't make it good. I don't care if OOXML was a standard, Mondopoint shoe sizes are also a standard. Nobody uses them. This type of pseudo-scientific claims that standards are eo ipso good is something I hate.
follow Download link for full PDF
they didn't publish this using Silverlight.
Have gnu, will travel.
The United States Army uses 'PureEdge' which i guess was replaced by IBM with 'Lotus Forms' as there is no canonical link to the software anymore. Its an XML based form system. Its not really used in any standard way, other than electronically saving forms, and filling stuff in before printing the forms. It could though, because the Army, at least, does little to no documentation that isn't on some kind of standardized form. Now that the forms are machine parsable, I can definitely see the fed adoption some kind of organization and retrieval system.
The problem with that is, that the government doesn't want to organize its documentation that well. Obfuscation is still a large part of information security in certain circles, and the possibility of leak is much greater when information flows so fluidly. Unclassified does not mean its not of a sensitive nature, it just means that it doesn't fall under any of the standard security classifications. Thus the reason why we shred EVERYTHING.
Its archaic, but not necessarily ineffective.
What would be a good start would be to standardise the publication of tabular data, for example population statistics, with ways of defining column types, data types and units whilst retaining a tabular structure instead of bastardising the tree-structure of XML. I guess we could take CSV, add a couple of header blocks and call it Extended CSV. Though it'd need an X in it to sound 21st century... so how about CSVX?
If anyone googles that my web site will go down in flames...
If you don't risk failure you don't risk success.