Slashdot Mirror
Data Sharing, Government Style
rowama writes "The Department of Homeland Security and the Justice Department have been collaborating to develop an XML-based model for data sharing. After less than a year since the initial release, in October 2005, the National Information Exchange Model (NIEM) 1.0 Beta is out. It's big, really big. There are no less than 9 namespaces and plans for future expansion. Contact your local government contractor, with resume in hand, and you may be one of the lucky developers to implement NIEM-capable software."
As an added bonus you can add a wee bit of code to make sure your name never ends up in these databases.
I Am My Own Worst Enemy
Meanwhile grandma is still taking off her shoes and getting wanded at the airport. Nice to know yet another debacle is launched. Here's hoping they're as successful as they have been with the new Air Traffic Control System.
Okay...I can see the need for u:SuperType->u:ActivityType->c:ActivityType->im:Al ienEncounterType...I mean, we're bound to encounter aliens at some point, right?
But im:AlienStudentDisciplinaryActionType? Planning for Alien encounters is one thing, but planning for dealing with them in our school systems seems like bureaucratic bloat to me. I don't think the Red Staters will be down with their taxes going to teach godless little green people.
(end humor tags)
ad logicam Claiming a proposition is false because it was presented as the conclusion of a fallacious argument.
The solution to every problem begging for more markup and extra layers of abstraction.
Neato. Maybe now they'll make less errors in that Terrorist Screening Database they have. You know, the one that has the names of over 250,000 people tagged as terrorists used in everything from no-fly lists to border crossings ever since the administration wanted all such watchlists to be consolidated into a single big one. That one the NSA probably uses. That one that, according to Department of Justice Inspector General reports, may be riddled with errors.
Read the Department of Justice and Department of Homeland Security Inspector General reports. They redact sensitive information in some cases, but based on context you can identify information in some places they've failed to redact in others. Keep on reading and you'll remember things to fit together a bigger picture.
National Information Exchange Model
On some level, information has always been exchanged between these powers. Now they're using XML. Cool.
Of was that a different Senate?
Lacking <sarcasm> tags,
Is the government following the Microsoft model of bloatware or it was it Microsoft following the government's model of bloatware. I keep getting the two mixed. There should only be one standard form of bloatware. I wouldn't be surprised if some OSS twits are starting their own bloatware.
The National Information Exchange Model (NIEM) 1.0 Beta is out. It's big, really big.
But that's peanuts to space.
Just glancing at it, I can see problems. XML is too often used for databases when it shouldn't be, but there are similarities, and just looking at it I can see that it violates one of the most basic database design principles: normalization
Just as an example, there are three different namespaces dedicated to the various FIPS (Federal Information Processing Standards)...To three different STANDARDS.
I'm no expert on government info, and I just looked at this thing for the first time, so maybe it's brilliant and I'm ust not seeing it, but it sure looks a lot like they've fallen victim to a database noob mistake, and created a monster tree with disproportionate crazy branches everywhere, and that is bound to cause relational problems, redundant data, and warped design challenges.
ad logicam Claiming a proposition is false because it was presented as the conclusion of a fallacious argument.
Which is worse, the government making their own format, or them using a microsoft one? I vote the latter. The format is open to public scrutiny, and that's a very good thing. I rather like this.
If nothing else this advances the idea that people should have control over the format they use. OK the people in this case is the government, but we shouldn't complain about anything that may improve communication of data.
It looks pretty competant too.
Time for Open Source people to leap forward and develop tools for it.
PURGE DATABASE
Happiness does not come from having much, but from being attached to little.
It's big, really big.
Yankee style rhetorics seems pretty stupid to me. Leaving out empty and pointless sentences improves readability and gives the reader a more positive image of the writer. Also, a neutral presentation would bring up the idea in a more efficient way.
XML schema by beuracracy.
This oughta be sweet.
Somebody oughta keep records on the SDLC of this thing. It should be enlightening in a huge trainwreck kind of way.
And there I was thinking TFA was about this:
http://news.bbc.co.uk/1/hi/business/5187276.stm
Is it just me or is this the first government website in CSS? ... and it just about validates!
You put entirely too much thought into that, despite it being completely off-topic.
I skimmed through the 58 page spec document which was mostly filled with describing the vast levels of bureaucracy that they're putting place to manage this beast. I also did a simple word find on the word "security". I only found 3 instances of the word that weren't coupled with the word "homeland" as in the Department of. No instances of the word "authentication".
I know this is doc isn't intended to show the exact structure of the messages to be passed, but gee whiz, wouldn't you think they would address the topic of "how do we make sure that only members with access to the NIEM can retrieve/exchange this data."????
All I found was a quick reference to one of the committees that they're going to form - who has the responsibility of (paraphrasing) "helping member organizations handle data security".
That's kinda scary. Or does this thing just run on the super-duper-secret world-wide government inter-network? I mean, they never have any problems with data security on that thing. (see: Los Alamos Lab, Dept. of Veteran Affairs, etc.)
I'd give you a plus one insightful if I had mod points.
>>Contact your local government contractor, with resume in hand, and you may be one of the lucky >>developers to implement NIEM-capable software. Lucky? When are good and decent programmers and other tech folks going to lay down their arms, so to speak, and refuse to further enable these turds? In an ideal world there would be a severe shortange of people who help enable the current US mal-administrations aims through technology. I'll starve before I help them commit any more crimes. Anyone else?
The most effective form of data sharing so far seems to have been the storage of that data on laptops leaving the building ...
Life sucks, but death doesn't put out at all. -- Thomas J. Kopp
XML is often poorly normalized such that you have to go out of your way to remove nesting, redundancies, etc. Why not clean up comma-delimited a bit so that it offers better meta-data (column types, etc.) and multi-schemas per file. Comma-delimited is also more compact. For some ideas, see:
o Xml
http://www.c2.com/cgi/wiki?RelationalAlternativeT
Table-ized A.I.
All the bloat will demand more storage, memory and faster cpus. With Vista late, XML is the only thing driving hardware sales these days. I used to say "XML is a solution looking for a problem". Now I understand the problem was flat hardware sales and XML is the solution.
Yes, it will end up being used as database using XQuery, or worse custom implemetations of similar beasts.
And yes, since it is not normalized data consistency will be lost leading to false psoitives and false negatives.
And since it is so bloated wait for a plethora of stripped down 'sub' versions with varying degrees of incompatibility.
Oh what a joy!
putting the 'B' in LGBTQ+
at this level? It's an XML data-interchange format. A stylesheet. It covers the format data has to be in to be transferred from system A in agency X to system B in agency Y, the security would be handled at the connection level, not the data level. When you log in to a website, slashdot for example, I doubt the CSS says much (if anything) about the security of the connection.
Best Slashdot Co
Either way, never understimate the power of the government to screw something up .
This space intentionally left (almost) blank.
Does this sound like NTAC operations in The 4400 to anyone?
You opened PersonName and closed it with PersonFullName ; your xml won't be parsed good like that ;)
--- I am known for the ones who want to find me on the net. Is that a privacy risk or a privilege? One might wonder..
I worked with the Global Justice XML Data Model, or GJXDM for short. It is endorsed by the DoJ. By endorsed I mean that the DoJ only gives out grant money to projects that use it. GJXDM is getting folded into NIEM. Anti-Terrorism is where the money is at from the feds, so everyone is chasing it. Anyway, my whole point to this post was to state that I would not consider myself 'Lucky' to work with NIEM XML specs. GJXDM is horrible. It is too large, too complex and too ambigious just to name a few problems. California did it's own version called the Second Generation Electronic Filing Standards, or 2GEFS. It's simple, well documented and easy to work with. They both are used to meet similar needs, but you can only use the federal version if you want grant money. To me, NIEM represents a HUGE waste of tax payer money. I have no doubt the specifications will be needlessly complex and technically inefficient. But that won't stop our government from wasting billions of dollars endorsing it!
I know you're joking (well, maybe), but I often think that the Slashdot crowd fails to appreciate how many people there are in the world -- very smart people, in fact, in many cases -- who are more than willing to take the "dirty" jobs.
Particularly if they're interesting dirty jobs.
The fact that what you're doing can be used to kill people fades away into relative unimportance pretty quickly, if there's a cool technical challenge to be solved, and the salary is right, and the people you get to work with are similarly goal-oriented.
There are a lot of people in the world who spend their days thinking of new and interesting ways to kill others, and I'm willing to bet that most of them probably don't lose sleep at night over it. The human mind has an amazing ability to rationalize -- if not flat-out ignore -- almost anything, and social mores regarding the value of others' lives are no exception.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
If your goal is to create yet still more bloated systems with yet still more arcane data constructs, this is a good start.
... why mess with a good thing?
Well, it's proved to be a pretty good business model so far
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
Statistically who is more likely to carry a weapon on an airplane? Grandma or Mohammed Habib over there?
t m
http://library.thinkquest.org/CR0212088/tertime.h
Only the State obtains its revenue by coercion. - Murray Rothbard
How can less errors help with a method that was completely flawed from the start? I mean, didn't it ever occur to anyone in DHS that normal, law-abiding American citizens might have names similar to at least some of those that are on "the list," and that because of this, they'd be subject to baseless abuse by those relying on it? After all, someone would never attempt to identify themselves as someone other than who they really are...nah, that would never happen.
Meanwhile, the FBI is still running the story that their computer system is screwed up. They've been running this story for 10 years. Is that believable in the context of this rollout?
Maybe they're using the same company that cost the IRS a few hundred million a few years ago- for something that was eventually scrapped. With the current "it's fer terrism" mentality, and all the illegal spying and end-runs around various "obstacles" imposed by the constitution, I'd hope for a repeat performance.
We are currently working with several government agencies that wish to expose data via the NIEM standard. The MetaMatrix product is being used to map current data sources into NIEM compliant views of that data without ever writing a line of code.
We have a NIEM specfic example that demonstrates this capability by using a pre-fab Derby database. Our product is downloadable for a free trial for anyone who might be interested. Here is the link to the example:
http://devcentral.metamatrix.com/products/examples /Home
I've worked on this a bit, and it has some real value despite it's obvious shortcomings. It is the case today that data exchange between different kinds of agencies essentially never happens, whereas within one flavor of agency (police, fire, ems, 9-1-1, emergency management, FEMA, .....), there often exists data interchange standards. I've been working on emergency calling (9-1-1) and I can tell you that the lack of data standards between different kinds of organizations really hurts, and some of us have some hope NIEM will help.
Don't knock it until you have been in the trenches and have a better answer.
Statistically who is more likely to carry a weapon on an airplane? Grandma or Mohammed Habib over there?
Considering grandma has dementia and a drug habit, I'm betting on grandma. That bitch is crazy!
Having the NIEM guide database design is a danger. But, if an agency models their database on it, it won't because they weren't warned against it. (Not sure if there's the correct number of negatives there. The point is that agencies are being explicitly warned against using the NIEM as the basis for their databases.)
Still, many make the initial assumption that the NIEM should guide their internal databases. Continual education is needed to prevent that misconception.
Regarding stripped-down versions, there is a mechanism, and supporting tool, for doing just that. It doesn't totally prevent the problem, but tries to keep it manageable.