Slashdot Mirror
NSA Mimics Google, Angers Senate
An anonymous reader writes "In a bizarre turn of events, the Senate would prefer that the DoD use software not written by the government for the government. Quoting: 'Like Google, the agency needed a way of storing and retrieving massive amounts of data across an army of servers, but it also needed extra tools for protecting all that data from prying eyes. They added 'cell level' software controls that could separate various classifications of data, ensuring that each user could only access the information they were authorized to access. It was a key part of the NSA’s effort to improve the security of its own networks. But the NSA also saw the database as something that could improve security across the federal government — and beyond. Last September, the agency open sourced its Google mimic, releasing the code as the Accumulo project. It's a common open source story — except that the Senate Armed Services Committee wants to put the brakes on the project. In a bill recently introduced on Capitol Hill, the committee questions whether Accumulo runs afoul of a government policy that prevents federal agencies from building their own software when they have access to commercial alternatives. The bill could ban the Department of Defense from using the NSA's database — and it could force the NSA to meld the project's security tools with other open source projects that mimic Google's BigTable.'"
This seems like a result of the conservative cry to shrink the size of the federal gubmint. "Gubmint shouldn't be allowed to do internally what they can outsource to some private company" possibly owned by China. THis is sad
Why should we get something for free when we can pay for it? Wait a minute....
The simple truth is that interstellar distances will not fit into the human imagination
- Douglas Adams
Accumulo runs afoul of a government policy that prevents federal agencies from building their own software when they have access to commercial alternatives
Just arrange to sell it to Google, make them the maintainers, and buy it back for $1.
All ideas^H^H^H^H^Hprocesses in this post are Patent Pending. (as well as the process of patenting all postings)
It is the result of private corporations lobbying for more privatisation. "Shrink the Government" is the voter-friendly PR spin on it. We have the same in the UK...fortunately the privatised "security" company G4S has just screwed up so massively that the agenda must have been put back a year or so. Personally, I think that any and all national security functions, whether physical or cyber, shouldn't be provided by anybody whose managers I cannot vote out of office.
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
for bills &c.
They're created w/ a tool named ACOMP.EXE (which the GPO used to use to make their style manual --- which typeset exactly like a printed copy I have from 1943 --- the new version is done w/ Adobe InDesign CS3 though).
If the Senate can use a special software tool for so prosaic a function, why can't other parts of the government?
William
(who recently had to download the successor to NIH (National Institute of Health) Image to make a reasonably-sized bitmap for placement into an automated pagination system when Adobe PhotoShop insisted on wrapping it up in all sorts of metadata, resulting in a several KB file, when JImage was able to write it out in a mere 480 bytes.)
Sphinx of black quartz, judge my vow.
"The bill indicates that Accumulo may violate OMB Circular A-130, a government policy that bars agencies from building software if itâ(TM)s less expensive to use commercial software thatâ(TM)s already available. And according to one congressional staffer who worked on the bill, this is indeed the case." Sounds like the alternatives to Accumulo are cheaper?
Several years ago when I was a young service member and working for around $25K a year to develop software for the military, I was told that the military was moving away from GOTS solutions and was mandating that everyone move to COTS software. They replaced my position with contractors that made $75K a year and ultimately with multi hundred million dollar contracts with contracting firms who "integrate" in COTS solutions. Granted having become one of those contractors myself and having over doubled my pay in that time frame, I do have to admit I appreciate that cheaper COTS solution. Though I do often times wonder to myself if the Government centralized their development efforts, tracked industry standards for producing secure code, and further developed some of the charming projects they have worked on (like SELinux) what the world would be like today. Just think, instead of knowing a huge ass hole is in your current revision of router code, you could simply send it off to the developers to repair. No lack of a $100K+ support contract to prevent you from getting a patch...
Select from tblFriends where interesting >= 4;
In a bill recently introduced on Capitol Hill, the committee questions whether Accumulo runs afoul of a government policy that prevents federal agencies from building their own software when they have access to commercial alternatives
I work at a large defense contractor, so obviously I'm posting anon. My thoughts on this are as follows: indeed there are requirements to use as much COTS and/or FOSS as possible for things that already exist (and so long as the use of any does not/cannot cause no future licensing issues that can be reasonably foreseen.)
Is in an effort to avoid the "not invented here" syndrome that plagues commercial and government enterprises alike. But the operative idea is that we should use a COTS if it provides the functionality that we need. If there is some type of deviation in the type of functionality that a project needs, it is perfectly reasonable to add new logic around it (or build one from scratch altogether.)
The NSA requirements for retrieving and storing massive amounts of data, when taken as is, do sound like something that Google already does. However, there are other requirements a Google-like COTS might or might not meet or might not meet efficiently (.ie. "tweaking the COTS will cause substantial operational costs down the road", just as a hypothetical example.)
There are needs to attach security label classifiers (TS,S,R,C,SBU,U), and compartment/silos to meet "need-to-know" requirements. There can be security-related non-functional requirements that say the mechanisms for storing/retrieving information above a certain security label be also be labeled with a classifier as strict as the data being handled. Part of the software system might be required to exist within Type 1 cryptography products, with physical shielding and all. It might be required to provide interfaces and protocols aware of sneakernet and airwalls.
Things like that do not get solved by deployment schemes and configuration alone. So "mimicking google" might not be descriptive to what's really going on here.
Furthermore, it looks incredibly stupid for Congress to be telling the NSA to shelve their own FOSS and to look for a COTS alternative. Sometimes, for some types of operations, you simply do not want a COTS. Fine for building government owned systems that handles, say, tax or immigration/nationalization records. Not so fine for TS-level material.
The NSA has been guilty of some major pork-barrel mishaps, and needs fiscal supervision. Hell, the whole defense sector is plagued by inefficiencies. However, this particular action by Congress, it's not a solution.
I suppose I'll be moderated "troll" if I suggest that the government shouldn't waste time and money rewriting software that already exists and can be licensed in the commercial market.
That isn't trolling at all. But I don't see why it shouldn't be handled like any other purchasing decision.
Commercial Product A cost $X
Commercial Product B cost $Y
Paying developers time to create that product will cost $Z
All else being equal, why _wouldn't_ you choose the option with the lowest cost?
Of course all else is rarely equal, but still people in companies do this kind of thing daily, weighing the cost vs benefit vs features and then factor in the other issues such as support/maintenance over the lifetime of the product and the computing resources required to use said product.
If paying developers to create it and maintain it turns out significantly cheaper than the other options, it only makes sense to create it in-house.
If buying it and paying the support contract, as well as paying for modification/customization of features turns out cheaper than other options, then it makes sense to buy the thing and not worry about it.
Without knowing dollar amounts involved and the required feature list, it's impossible to know what each option costs in whole.
We also don't really know all the factors involved. I'm sure cost is a factor in there somewhere, but it could rank anywhere from #1 to #last.
Clearly, someone must have paid for this charming little legislative tidbit. But who?
I mean, I could understand if Lockheed-Martin had a proprietary solution that they were offering (with just a few change orders needed to satisfy NSA's requirements, of course), but the beneficiaries here seem to be the Cassandra and HBase projects, neither of which seem likely to have much of a lobbying budget. Was it their forebears at Facebook? Could they possibly care enough?
And blaming it on "conservatives-want-smaller-government" seems pretty silly, too. Sure, turfing Accumulo might conceivably further that goal in some tiny, tiny way, but it's not like some senator was likely to have figured this out by himself. No, clearly someone put them on to it, but who and why?
It's an intriguing mystery. Any ideas?
...the committee questions whether Accumulo runs afoul of a government policy that prevents federal agencies from building their own software when they have access to commercial alternatives.
Is this a joke?
I thought Doug Cutting, creator of Hadoop, did a lot of the work on Accumulo too. And they open-sourced it for more people to use, how can that possibly be bad? This seems backwards, it seems the NSA is doing something good here in making up some nice software and releasing it to the world. I think the real question is what sort of vested interest these senators have in the businesses that would "sell similar technology" to the gov't.
Vertically integrating your own software stack isn't necessarily a bad idea. At some scale, if you have enough internal resources, supporting your own code stack becomes more effective than dealing with a large number of third party contractors that are often competing with each-other and not 100% mission focused (think profit motivation). While it makes sense to use a COTS (commercial-of-the-shelf) application for certain problems, the problem of National Security I don't think should be corporatized. I think they should be using the best tools, whether internal or externally developed.
Remember $500 hammers? Back in the 1980s, there was a big push to reduce government purchasing costs, especially for military projects, through the use of "Commercial Off-The-Shelf" technology, so whenever possible you'd buy COTS products instead of specially-made customized government-market products. It didn't always make sense, but in many cases it could save a huge amount of money, and realistically a large fraction of the stuff the government bought had commercial equivalents that already had economies of scale keeping the costs down. Sometimes the hammer costs $500 because it's made of MIL-SPEC Titanium, sometimes it's because you spend $490 setting up your hammer-making machine to run off two Left-Handed Jet Engine Hammers for the Air Force, sometimes it's because you spend $600 in contact-lawyer time writing an addendum to a ten-year-old contract to sell two more off-the-shelf hammers to replace the MIL-SPEC ones that got lost.
Government procurement has always had a lot of "check the box on the contract" requirements. Sometimes they make sense, like using COTS to save money when there are commercial products available (especially if that means forcing the organization that wants the stuff to be realistic about what they need.) Sometimes they're theoretically required, but in practice the agency can get a waiver (so everything needs IPv6, but they actually use IPv4, and POSIX was required from mid-80s on but everybody got a waiver and used MS-DOS for office equipment.) Sometimes they increase the costs because the purchasing department puts all that stuff in the contract even though the users don't actually need it.
I did work on some projects where COTS didn't make sense. We were bidding on a communications system that used X.25 (which wasn't yet obsolete :-), but the civilian agency that wanted it had asked the NSA for help specifying a system that would be secure. So yes, it was X.25, but with dozens of special options that no commercial equipment used more than a few of. And the contract specified COTS. How do you reconcile the problem and let the agency check off the "COTS" box on their contract? Make the device, offer it for sale to the market, have a couple of your subcontractors buy boxes from you for "testing" or "evaluation".
Another part of that project not only wanted special-flavor X.25 off the shelf, and POSIX, but also wanted a B1-secure operating system (but it was communication gear, so it would have to be Red Book B1, which was still way-future research, and we had one of the first Orange Book B1 Unix boxes), and GOSIP (the OSI networking stack, though nobody had a GOSIP stack that worked with that particular flavor of X.25 options.) A later project I worked on wanted B1 Secure, POSIX, Ada, POSIX Real-Time (even though the spec wasn't baked yet, and the B1 Secure Unix system didn't support it, and getting that re-evaluated would cost $250K even if we could figure out how to make it work :-)
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks