Slashdot Mirror
FBI's New Info-Sharing Software Project Fails
Spy Handler writes "After 4 years and half a billion dollars, FBI's attempt to create new information sharing software - called Virtual Case File - simply didn't work.
← Back to Stories (view on slashdot.org)
To the Feds -
Come look for me the next time you need something that does not work. I would be happy to deliver it for the bargain basement price of only $100,000,000.
Inconceivable!
Since the attacks, Congress has given the FBI a blank check, allocating billions of dollars in additional funding.
And that blank cheque is the problem. Whatever happened to accountability? It's the tax payers money to begin with.
Free XBox, PS2
Having worked at Andersen Consulting little more than a decade ago and seeing the dismal IT failures of EDS has had in England, when I here of vast amounts of money wasted on failed IT projects these companies immediately come to mind.
Perhaps you should try this?
:)
Hm, more seriously.. They must really have tried to make something special. Otherwise WebDAV+SSL would have proven to be a bit cheaper..
Probable impossibilities are to be preferred to improbable possibilities.
Aristotele
From a position of a tax payer this frustrates me.
However, as a programmer I can understand them wanting to scrap the program. If the design has been shot to hell, if their using technology several years past its prime, it's time to start fresh.
And as a tax payer, I'd prefer the FBI to use a system that works, rather than a system that doesn't.
-Teiresias
... The MPAA and RIAA have filed suit against the FBI, since their software could be used to share copyrighted material.
But God demonstrates his love for us, in that while we were yet sinners, Christ died for us - (Romans 5:8)
I actually read the FA (Ok, scanned it), and I didn't see anything that the FBI required that isn't commonly available now. Get a robust DB, have information decrypted at the user's computer, do not have any portion of this network on the Internet - instead use VPN/SSH connections physically secure the boxes, etc. Why they went to a third party in SD who blew through 130 MILLION of our tax dollars with nothing to show for it is beyond me.
"As the intrepid kobold companion continues his journey, he begins to wonder... if priests raises dead, why anybody die?
Too bad
Trolling is a art,
This sounds like a perfect project for the open source community! We should get Richard Stallman to submit a quote.
Or better yet, Bram Cohen (inventor of BitTorrent) or Jed McCaleb (inventor of eDonkey). Those guys have experience writing file sharing programs after all, and isn't that really what the FBI is asking for?
It's hard to soar like an eagle when you're surrounded by turkeys.
From TFA: So far the overhaul has cost $581 million, and the software problems are expected to set off a debate over how well the bureau has been spending those dollars.
I'm going to go with "not well."
One man's -1 Flamebait is another man's +5 Funny.
If the FBI had released its information encrypted as Metallica MP3 files, it would have been a resounding success.
Don't blame Durga. I voted for Centauri.
WASHINGTON -- A new FBI (news - web sites) computer program designed to help agents share information to ward off terrorist attacks may have to be scrapped, the agency has concluded, forcing a further delay in a four-year, half-billion-dollar overhaul of its antiquated computer system.
The half-billion is entire their budget to overhaul computer systems, not how much money they spent on this software.
This is not to say they haven't wasted any money:
Science Applications has received about $170 million from the FBI for its work on the project. Sources said about $100 million of that would be essentially lost if the FBI were to scrap the software.
Sometimes it best to scrap a software project and take the lessons learned from the failure to a new project. As long as knowledge was gained about why the project failed then not all was lost.
Can I bum you a
FBI can just post their information to usenet group gov.fbi.bigsecret and then use google to search for needed information!
Now gimmi my half a billion dollars please! (I do accept paypal)!
I don't recall having any involvement with this project.
Someone at the FBI definitely should have read this article.
-Rob
Marriage doesn't have to suck!
When I worked for the Department of Justice, a case might have 5 different case numbers: one case number for the DOJ, one case number for the FBI, one case number for the Defense Criminal Investigative Service, one case number for the Naval Criminal Investigative Service, etc. If I only had the DOJ jacket number, it could take me 15 minutes to get the case number for another agency, just so I could talk to one of the investigating agents.
Spend money to fix that larger-scale problem, before flooding the FBI with money to squander on a software application that they will be terminating and starting afresh on.
The bureau is no longer saying when the project, originally scheduled for completion by the end of 2003, might be finished. ...
...
...
A prototype of the Virtual Case File was delivered to the FBI last month by Science Applications International Corp. of San Diego. But bureau officials consider it inadequate and already outdated, and are using it mainly on a trial basis to glean information from users that will be incorporated in a new design.
Science Applications has received about $170 million from the FBI for its work on the project.
A spokesman for Science Applications, Ron ollars, said via e-mail that the company had "successfully completed" delivery of the initial version of the Virtual Case File software last month.
The stripped-down prototype will be running for three months. The bureau plans to then "shut it down, take all the lessons learned and incorporate them in a future case management system," a person familiar with the bureau's plans said.
An outside computer analyst who has studied the FBI's technology efforts said the agency's problem is that its officials thought they could get it right the first time. "That never happens with anybody," he said.
Some sources sympathetic to the FBI defended the process, and said that what has been learned in designing the software has given the bureau
valuable design and user information.
The first time they saw the software was a year after the delivery date. So they must have been using waterfall. Then they defend the process by saying the only good thing they got out of it was the information for the next pass of iterative development. So the best thing about waterfall is that when it fails you can turn it into iterative. Pure genius.
Stop-Prism.org: Opt Out of Surveillance
This is a prime example why public funded software ought to be open source, that way the community as a whole can pick bits and pieces out of it for further use.
"The likes of Facebook and WhatsApp are free to those whose privacy is of zero value."
Perhaps using the Duke Nukem engine as a front end was a bad idea?
Seriously, when you look around it's amazing how many software projects just completely fail with no usable code produced. It's not uncommon for companies to spend several million and just shut the thing down a couple years into it.
I think we're about a century behind our technology. We still try to use industrial age models for 'building' things...and the digital/info/[buzzword] age has major implications that those models just don't take into account.
Well, in TFA, it says that the program was contracted out to a company named Science Applications.
It's impossible to tell what the problem was until we learn more. As someone who works in government contracting, I know that it can be like pulling teeth trying to figure out WTF the government actually wants. I'm sure a typical requirements review went something like:
ScienceApps: So, what do you guys want?
FBI: You know, a case management system that does stuff.
ScienceApps: Care to elaborate?
FBI: Sorry, gotta go.
Why don't they do the same for software?
These are the same feds who treat copyright infringement as "theft"; who tack on all sorts of costs to the cost associated with a breakin (where a kid just pokes around the system); and yet they turn the other cheek when these companies waste billions of dollars on badly-executed projects.
As a taxpayer, I am thoroughly pissed at this waste of my money.
Expect the Prime to pay a token couple of million dollars as a "fine" and walk laughing all the way to the bank...
Sounds just like the TIERS Texas Integrated Eligibility Redesign System software my agency has been trying develop. The Texas Department of Human Services, now the Texas Health and Human Services Commission, has contracted to Deloitte to develop a web based system similar to what is described in the article. $3 million a month (according to some) has been spent on this for a couple of years now and it is a HORRIBLE excuse of a system. I know case workers that are being forced to test the software that say it takes at LEAST twice as long to work a case now than it did with the old system that was developed in the 80's. This has been a boondoggle in the worst sense and any Texas taxpayer should be pissed off about it.
It gets to be depressing working for the government because you see so many contracts like this awarded simply because some higher up gets his palm greased. Another example of this is the fact that I had to pay Banctec (the company that has our hardware support contract) the standard fee of $340 to replace a CPU FAN in an old machine the other day. So sad.
P.S. - I'm having to post this anonymously because anyone that has even begun to criticize the TIERS software, even internally, has been officially reprimanded or worse.
Lots of people are saying "How could they spend that much money on software that doesn't work". Clearly you've never worked on a government project. I'm working on one for a federal agency. We are migrating a small piece of a small department of this agency from legacy systems (mix of mainframe, access apps, excel spreadsheets, and 3x5 cards) to a J2EE solution. We are 4 years in with an average of 40 consultants at any given time. That alone is 16 million dollars, roughly, not including hardware and software. And this is for a very small set of features just related to storing client information and tracking the state of financial transactions.
We're less than halfway done. The requirements, when we can get any, are both vague and constantly changing. Direct contradictions are the norm, and anything wildly illogical (e.g. "If a transaction is partially rolled back, we still want the state to show that it was completed, except when we don't, and we can't define when we don't.") is justified with "because I said so". As contractors, we are not allowed to question anything the government people say no matter how clearly wrong it is. Several people have been fired for asking to have requirements clarified.
This is how they spend hundreds of millions of dollars on a system that doesn't work.
Just reading the article makes me think this was more a problem of trying to change the way the FBI works through software instead of making fundamental changes in the way they manage their people.
****
"I'd never want to join a club that would have me as a member" - G. Marx
Sir,
Your offer has been accepted. You are invited to become part of our Guantanomo Detainee Program (which currently does not produce intel or results).
Please report to your nearest police station for arrest and processing. No personal effects allowed (as your living space is significantly restricted).
Thank You
FBI
There is always a frontier where there is an open and willing mind
And the scariest thing is that they're looking to BAE Systems next. I have (well, had) family there for a while. If their experience is anywhere close to typical, this is not an organisation you want to help you run critical security/intelligence concerns.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
I guess a Wiki wouldn't work for them... Blogs wouldn't work either. A copy of the Code that runs SLashdog with heads of major field offices playing the Role of Cowboy Neal wouldn' work either..
Ok, my point is now that they have a secure network there are some many great ways to share data, and even rank and Meta-moderate data...
Sure it's nice to build some amazing wild system that totally solves every problem they ever had and ever will have... BUT there is too much risk.
You see this happening again and again in Government, with FBI, IRS, etc. Big huge systems build from whole cloth rarely ever fit or work or are delivered as promised.
Smaller systems with continual or incremental changes work better.
http://www.hawknest.com/
According to the article, $170 million has been paid for software development. $100 million of that will be lost of the software is scrapped. The rest went to purchase thousands of computers and set up new networks.
So, surprise, the slashdot story title is misleading.
Dan East
Better known as 318230.
i think you fail to see how a gov project works.
steps to gov project.
1. create team of 4-5 people to outline requirements
2. get team staff of 2-3 to support them
3. get place for them to work for up to 2-3 years developing requirements
4. subcontract out requirements analysis to someone
5. hire subcontractor to verify requirements analysis by the first.
6. hire gov people to oversee both contractors
7. hire people to support people overseeing subs.
8. release requirements out for public "auction"
9. review company responses to "auction" by s team of 12-20 people
10. hire crew to support 12-20 people on the responses review.
11. except bid.
12 start project.
there you have approximately 100-200 people working on just ensuring the requirements documentation and bid for 3-4 years before a project even starts. that alone could cost 200 mil or more.
they got off cheap.
FBI wants a computer system where the officer can sit down at terminal with donut in hand, type in the name of the crime, and get a popup list of all the folks guilty of that crime.
Police work Chief Wiggum style.
liqbase
One of the hardest things in working for government is that in order to write software properly, you need to get a good look at the data you're working with. You can't see this data; it's heavily, heavily classified.
It's classified two ways: first, a lot of this data is privacy protected (the FBI spies on American citizens and that data is heavily controlled). Second, one of the things it needs to store is sources&methods, which are protected even more closely than the data itself. (The most classified stuff is always about sources&methods, not the data itself.)
The open-source community could write pieces of it, but the hard work on a project like this is adapting it to the particular requirements of the customer.
The problems involved aren't abstract ones that can be solved byu an incredibly clever person like Bram Cohen. They're involved in getting a gazillion people to all buy off on a data format, and convincing them that they really can share information without violating their security requirements (which is really just code-speak for "if I let you have this information I won't be the only one with it, and therefore I become less important.")
The security clearance requirement means that they're working with a drastically reduced pool of programmers. Corners get cut, ideas go unused for lack of implementers, internal oversight is practically nil. (They have code reviews but they're an immense waste of time.)
I'm not sure I've ever worked on a government project of even a tenth this size that I considered to be successful, even if it did get deployed. But throwing it out to the open-source community isn't an option.
If it seems too impossible to bungle a project like this...it probably is. I'd bet the half a billion has been funnelled to some covert program.
Your tax dollars at work....
Sometimes seventeen/Syllables aren't enough to/Express a complete
Imagine the learning curve! Here we have this monolythic blob of stuff that can do EVERYTHING. Here are the 12 billion source lines of code and a make file that takes 3 centuries to run. The project is 4 years behind schedule and we fired the 10,000 coders that were working on it before, but we asked them to comment their code with nice flower boxes.
From my experience, the bigger the project the more likely it is to fail. Making lots of little bits out of one big one may result in some integration hiccups, but at least there will be useful pieces and refactoring can be addressed on a priority basis.
When the people fear their government, there is tyranny; when the government fears the people, there is liberty.
Those systems generally work, more or less (more like "somewhat less", but enough to be worth it). When SAP fails to deliver a 7-8 figure project to a customer with a 10-12 figure annual budget, hundreds of millions of its own customers, and a global reach in vendors and operations, it doesn't get paid. Probably, it gets sued. The FBI should be good at that, but they seem more inclined to give away the money, without getting any justice. That suggests that some decisionmakers at the FBI, or above, are getting something pretty good out of the catastrophic deal.
--
make install -not war
I'd like a nice on-line form associated with my tax return, which allows me to pick exactly where my tax dollars are spent.
I'd tick off things like, "Sidewalks and road repair in the region of my choice."
I'd tick off, "Public Transportation."
I'd tick off, "Environmental Conservation" and "Well-funded Free Medical Clinics which employ doctors who really want to heal people and not just get rich." I'd tick off all the other things I want MY money to be spent on. I want to be able to micro-manage where my tax dollars go, what salaries people receive, and who gets to have a job funded with MY money.
Things I'd NOT tick off would include,
"Missile defense systems which A) don't work and B) increase world tensions leading to hugely wasteful expenditures on ever increasingly complex defenses. Which don't work."
"Spineless Yes-Man Politicians More Interested in Keeping Their Jobs than in Serving the People who Bloody Voted for them."
"Free Handouts and Make-Work Contracts for Stupid Corporations Which Don't Deserve Jack Shit, *cough Haliburton* But which Happen to be run by Friends and/or Family Members of Sitting Retard Presidents."
"Education systems which make kids stupid, socially retarded and massively mis-informed."
"Legal and Penal systems which put non-criminals into jails which are designed to shove everybody into beast-mode and encourage them to abuse one another just so that they might survive."
And of course,
"Half-Billion Dollar mis-leadingly named information-consolidation contracts which duplicate other contracts and existing systems which already work a bit too well at putting non-criminals into jail."
If I can't have that tax system, then I'd rather see the whole goddamned thing burn to the ground.
But hey, that's just me.
-FL
ScienceApps: What does the current system do?
FBI: You know, FBI stuff. By the way, we need a detailed budget and schedule by next week.
Mea navis aericumbens anguillis abundat
When will people learn: with extremely complicated systems that humans have to interact with you can not specify it 100% correctly the first time?
Experience in building such systems has lead many of us to realize you must use an iterative approach that allows the end users to be part of the feedback loop.
Release early and release often, let your users use and break the application, and come closer to the ideal system with each iteration.
Now, I wouldn't blame the FBI for the problem completely - after all, they are not software developers. A portion of blame should go toward the contractor for failing to realize the issues surrounding development of such a complex system and taking appropriate actions to determine and meet the needs of their clients. Their contract should have been written to a) specify customer satisfaction as the key measurable for success, and payment of the contract b) put in a rider that basically states any functionality needed to bring the application to minimal usability as discovery occurs will be part of the first contract (this is negotiable - some things are really enhancements and new functionality - and some are required, even though not originally discovered in the first iteration - this allows both parties to recognize up front that 100% discovery of requirements does not take place in practice).
This approach has worked extremely well for me as a manager of vendor development (I have been extremely lucky to have vendors who understand what I am talking about), as well as for my own projects that I develop and implement. While there is a bit of risk involved in negotiating key usability issues discovered late in the development cycle - going out of the gate with an iterative approach ameliorates much of that - and is certainly less risky than giving someone $100,000,000 before I see the first line of code...
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
During the 15 years I ran my own computer consulting business it was common to be invited to make a bid, do the analysis and present a proposal, only to have the analysis given to a another to impliment. Sometimes the connection was nepotism, sometimes it was a competitor who under bid, so the putative client thought they'd save money by using the low bidder. They "Cherry Picked" me. That happened only a few times before I realized what was happening and begin charging for the analysis. If they wouldn't agree to pay for the anlaysis I wouldn't submit a proposal.
I am wondering if a similar thing isn't happening here. SAIC is, in effect, being paid to the system analysis, but the most lucrative part of the project will be given to an insider, a crony or for a political payoff.
Running with Linux for over 20 years!
Jeez, they should have just saved Napster the first time.