Slashdot Mirror
FBI's New Info-Sharing Software Project Fails
Spy Handler writes "After 4 years and half a billion dollars, FBI's attempt to create new information sharing software - called Virtual Case File - simply didn't work.
← Back to Stories (view on slashdot.org)
Since the attacks, Congress has given the FBI a blank check, allocating billions of dollars in additional funding.
And that blank cheque is the problem. Whatever happened to accountability? It's the tax payers money to begin with.
Free XBox, PS2
Having worked at Andersen Consulting little more than a decade ago and seeing the dismal IT failures of EDS has had in England, when I here of vast amounts of money wasted on failed IT projects these companies immediately come to mind.
Perhaps you should try this?
:)
Hm, more seriously.. They must really have tried to make something special. Otherwise WebDAV+SSL would have proven to be a bit cheaper..
Probable impossibilities are to be preferred to improbable possibilities.
Aristotele
From a position of a tax payer this frustrates me.
However, as a programmer I can understand them wanting to scrap the program. If the design has been shot to hell, if their using technology several years past its prime, it's time to start fresh.
And as a tax payer, I'd prefer the FBI to use a system that works, rather than a system that doesn't.
-Teiresias
This sounds like a perfect project for the open source community! We should get Richard Stallman to submit a quote.
Or better yet, Bram Cohen (inventor of BitTorrent) or Jed McCaleb (inventor of eDonkey). Those guys have experience writing file sharing programs after all, and isn't that really what the FBI is asking for?
It's hard to soar like an eagle when you're surrounded by turkeys.
Sometimes it best to scrap a software project and take the lessons learned from the failure to a new project. As long as knowledge was gained about why the project failed then not all was lost.
Can I bum you a
The article doesn't really mention why they think it's going to fail. It seems like maybe the FBI didn't really know what it wants, and probably still doesn't know what it wants. Science Application, the company developing the program, has received about $170 million dollars for the project. But the article says the FBI has spent about $500 million on the project. Where did the rest of the money go?
Portland, North Dakota Puppies
When I worked for the Department of Justice, a case might have 5 different case numbers: one case number for the DOJ, one case number for the FBI, one case number for the Defense Criminal Investigative Service, one case number for the Naval Criminal Investigative Service, etc. If I only had the DOJ jacket number, it could take me 15 minutes to get the case number for another agency, just so I could talk to one of the investigating agents.
Spend money to fix that larger-scale problem, before flooding the FBI with money to squander on a software application that they will be terminating and starting afresh on.
The bureau is no longer saying when the project, originally scheduled for completion by the end of 2003, might be finished. ...
...
...
A prototype of the Virtual Case File was delivered to the FBI last month by Science Applications International Corp. of San Diego. But bureau officials consider it inadequate and already outdated, and are using it mainly on a trial basis to glean information from users that will be incorporated in a new design.
Science Applications has received about $170 million from the FBI for its work on the project.
A spokesman for Science Applications, Ron ollars, said via e-mail that the company had "successfully completed" delivery of the initial version of the Virtual Case File software last month.
The stripped-down prototype will be running for three months. The bureau plans to then "shut it down, take all the lessons learned and incorporate them in a future case management system," a person familiar with the bureau's plans said.
An outside computer analyst who has studied the FBI's technology efforts said the agency's problem is that its officials thought they could get it right the first time. "That never happens with anybody," he said.
Some sources sympathetic to the FBI defended the process, and said that what has been learned in designing the software has given the bureau
valuable design and user information.
The first time they saw the software was a year after the delivery date. So they must have been using waterfall. Then they defend the process by saying the only good thing they got out of it was the information for the next pass of iterative development. So the best thing about waterfall is that when it fails you can turn it into iterative. Pure genius.
Stop-Prism.org: Opt Out of Surveillance
Why don't they do the same for software?
These are the same feds who treat copyright infringement as "theft"; who tack on all sorts of costs to the cost associated with a breakin (where a kid just pokes around the system); and yet they turn the other cheek when these companies waste billions of dollars on badly-executed projects.
As a taxpayer, I am thoroughly pissed at this waste of my money.
Expect the Prime to pay a token couple of million dollars as a "fine" and walk laughing all the way to the bank...
Lots of people are saying "How could they spend that much money on software that doesn't work". Clearly you've never worked on a government project. I'm working on one for a federal agency. We are migrating a small piece of a small department of this agency from legacy systems (mix of mainframe, access apps, excel spreadsheets, and 3x5 cards) to a J2EE solution. We are 4 years in with an average of 40 consultants at any given time. That alone is 16 million dollars, roughly, not including hardware and software. And this is for a very small set of features just related to storing client information and tracking the state of financial transactions.
We're less than halfway done. The requirements, when we can get any, are both vague and constantly changing. Direct contradictions are the norm, and anything wildly illogical (e.g. "If a transaction is partially rolled back, we still want the state to show that it was completed, except when we don't, and we can't define when we don't.") is justified with "because I said so". As contractors, we are not allowed to question anything the government people say no matter how clearly wrong it is. Several people have been fired for asking to have requirements clarified.
This is how they spend hundreds of millions of dollars on a system that doesn't work.
Just reading the article makes me think this was more a problem of trying to change the way the FBI works through software instead of making fundamental changes in the way they manage their people.
****
"I'd never want to join a club that would have me as a member" - G. Marx
And the scariest thing is that they're looking to BAE Systems next. I have (well, had) family there for a while. If their experience is anywhere close to typical, this is not an organisation you want to help you run critical security/intelligence concerns.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
I guess a Wiki wouldn't work for them... Blogs wouldn't work either. A copy of the Code that runs SLashdog with heads of major field offices playing the Role of Cowboy Neal wouldn' work either..
Ok, my point is now that they have a secure network there are some many great ways to share data, and even rank and Meta-moderate data...
Sure it's nice to build some amazing wild system that totally solves every problem they ever had and ever will have... BUT there is too much risk.
You see this happening again and again in Government, with FBI, IRS, etc. Big huge systems build from whole cloth rarely ever fit or work or are delivered as promised.
Smaller systems with continual or incremental changes work better.
http://www.hawknest.com/
i think you fail to see how a gov project works.
steps to gov project.
1. create team of 4-5 people to outline requirements
2. get team staff of 2-3 to support them
3. get place for them to work for up to 2-3 years developing requirements
4. subcontract out requirements analysis to someone
5. hire subcontractor to verify requirements analysis by the first.
6. hire gov people to oversee both contractors
7. hire people to support people overseeing subs.
8. release requirements out for public "auction"
9. review company responses to "auction" by s team of 12-20 people
10. hire crew to support 12-20 people on the responses review.
11. except bid.
12 start project.
there you have approximately 100-200 people working on just ensuring the requirements documentation and bid for 3-4 years before a project even starts. that alone could cost 200 mil or more.
they got off cheap.
In theory there's a guy involved in the process who reads the bids and rejects those that seem infeasible.
In practice this guy is a manager, not a software expert, and he's usually an idiot. I've written dozens of proposals and it's monumentally clear that your job is to impress this idiot. Coming up with an intelligent design is something you spend time on after the bid, not before. And there's usually not time then, because you're busy fulfilling this idiot's pipe dreams.
Not that I'm bitter or anything.
One of the hardest things in working for government is that in order to write software properly, you need to get a good look at the data you're working with. You can't see this data; it's heavily, heavily classified.
It's classified two ways: first, a lot of this data is privacy protected (the FBI spies on American citizens and that data is heavily controlled). Second, one of the things it needs to store is sources&methods, which are protected even more closely than the data itself. (The most classified stuff is always about sources&methods, not the data itself.)
The open-source community could write pieces of it, but the hard work on a project like this is adapting it to the particular requirements of the customer.
The problems involved aren't abstract ones that can be solved byu an incredibly clever person like Bram Cohen. They're involved in getting a gazillion people to all buy off on a data format, and convincing them that they really can share information without violating their security requirements (which is really just code-speak for "if I let you have this information I won't be the only one with it, and therefore I become less important.")
The security clearance requirement means that they're working with a drastically reduced pool of programmers. Corners get cut, ideas go unused for lack of implementers, internal oversight is practically nil. (They have code reviews but they're an immense waste of time.)
I'm not sure I've ever worked on a government project of even a tenth this size that I considered to be successful, even if it did get deployed. But throwing it out to the open-source community isn't an option.
If it seems too impossible to bungle a project like this...it probably is. I'd bet the half a billion has been funnelled to some covert program.
Your tax dollars at work....
Sometimes seventeen/Syllables aren't enough to/Express a complete
Imagine the learning curve! Here we have this monolythic blob of stuff that can do EVERYTHING. Here are the 12 billion source lines of code and a make file that takes 3 centuries to run. The project is 4 years behind schedule and we fired the 10,000 coders that were working on it before, but we asked them to comment their code with nice flower boxes.
From my experience, the bigger the project the more likely it is to fail. Making lots of little bits out of one big one may result in some integration hiccups, but at least there will be useful pieces and refactoring can be addressed on a priority basis.
When the people fear their government, there is tyranny; when the government fears the people, there is liberty.
Those systems generally work, more or less (more like "somewhat less", but enough to be worth it). When SAP fails to deliver a 7-8 figure project to a customer with a 10-12 figure annual budget, hundreds of millions of its own customers, and a global reach in vendors and operations, it doesn't get paid. Probably, it gets sued. The FBI should be good at that, but they seem more inclined to give away the money, without getting any justice. That suggests that some decisionmakers at the FBI, or above, are getting something pretty good out of the catastrophic deal.
--
make install -not war
I'd like a nice on-line form associated with my tax return, which allows me to pick exactly where my tax dollars are spent.
I'd tick off things like, "Sidewalks and road repair in the region of my choice."
I'd tick off, "Public Transportation."
I'd tick off, "Environmental Conservation" and "Well-funded Free Medical Clinics which employ doctors who really want to heal people and not just get rich." I'd tick off all the other things I want MY money to be spent on. I want to be able to micro-manage where my tax dollars go, what salaries people receive, and who gets to have a job funded with MY money.
Things I'd NOT tick off would include,
"Missile defense systems which A) don't work and B) increase world tensions leading to hugely wasteful expenditures on ever increasingly complex defenses. Which don't work."
"Spineless Yes-Man Politicians More Interested in Keeping Their Jobs than in Serving the People who Bloody Voted for them."
"Free Handouts and Make-Work Contracts for Stupid Corporations Which Don't Deserve Jack Shit, *cough Haliburton* But which Happen to be run by Friends and/or Family Members of Sitting Retard Presidents."
"Education systems which make kids stupid, socially retarded and massively mis-informed."
"Legal and Penal systems which put non-criminals into jails which are designed to shove everybody into beast-mode and encourage them to abuse one another just so that they might survive."
And of course,
"Half-Billion Dollar mis-leadingly named information-consolidation contracts which duplicate other contracts and existing systems which already work a bit too well at putting non-criminals into jail."
If I can't have that tax system, then I'd rather see the whole goddamned thing burn to the ground.
But hey, that's just me.
-FL