Slashdot Mirror


The FBI Software Upgrade That Wasn't

Davemania writes "Washington Post reports that the FBI's attempt to modernize their department has once again failed. The 170 million dollar Virtual case File system, the agency's second attempt to go paperless is reported to be useless. The finger seems to be pointing at the FBI leadership, greedy contractors and bad software management." From the article: "It appeared to work beautifully. Until Azmi, now the FBI's technology chief, asked about the error rate. Software problem reports, or SPRs, numbered in the hundreds, Azmi recalled in an interview. The problems were multiplying as engineers continued to run tests. Scores of basic functions had yet to be analyzed. 'A month before delivery, you don't have SPRs,' Azmi said. 'You're making things pretty. . . . You're changing colors.'"

23 of 381 comments (clear)

  1. I love you by neonprimetime · · Score: 4, Informative

    I love helping you /.ers out. Instead of spending painstaking hours clicking thru multiple page news stories, I sit here and quickly provide you with printer friendly links

  2. Government Contract$ by Billosaur · · Score: 4, Informative
    But the problems were not the FBI's alone. Because of an open-ended contract with few safeguards, SAIC reaped more than $100 million as the project became bigger and more complicated, even though its software never worked properly. The company continued to meet the bureau's requests, accepting payments despite clear signs that the FBI's approach to the project was badly flawed, according to people who were involved in the project or later reviewed it for the government.

    And that is how you get rich doing work for the government. The government agency comes up with a half-assed plan, you put in a low bid, they accept and start handing you checks, and you make things look pretty, all the while hiding the flaws. In then end, you've become rich, the goverment runs a deficit, and the American taxpayer foots the bill.

    --
    GetOuttaMySpace - The Anti-Social Network
    1. Re:Government Contract$ by climbing · · Score: 2, Informative

      I have some experience in Government acquisition management and some of these /. rants (when temperred) are TRUE:
        - some contractors can bid low and get well on mods.
        - some large programs are constantly overrun.

      And sure you can say this or that should be outsourced and the people managing these programs need to be smarter, but here are the real problems IMHO:
        - the requirements generation and acqusition processes are BROKEN. It takes literaly years to document and approve the need, the capability, performance parameters, etc until you are eventually allowed to let a contract. the buerocracy pendulum is currently pegged. it's just too slow.
        - the user reps often end up documenting "solutions" instead of "requirements" (e.g. i want a U P-38 I saw at a trade show instead of I want a handheld unit with these constraints capable of...). or worse *sigh* a congressmen sets aside money in your agency's budget for his/her favorite company irregardless of agency need. that's always fun.
        - the financial management processes are BROKEN. The budget is on a 4 year (new starts) / 2 year (tweaks) cycle. Right now you are setting the final budget for FY2010/08. more importantly, there are no incentives for the purchasing command to save money. In fact there can be penalties. E.g. 10M per year program. You get crafty and get it all done for 8M + 1M incentive bonus = 9M in year one. It will be incredibly difficult to avoid having your budget cut at the miodyear or year end reviews because you will be way behind in your "obligation and expenditure benchmarks." Expect at least a $1M recurring cut per year thereafter. The system incentivizes full or over-spending; not savings.

      Reality check:
        - there are some very hard working people in the government... swimming upstream. the succeess stories rarely make the headlines. here's one. we put together a 5 year global telecom contract for an agency that saved 10%-20% per year and brought the system availability from ~90% with many single points fo failure to 99% with prioritized redundant failover. um, well no... it wasn't at the FBI.

  3. Re:Paranoia! by acroyear · · Score: 3, Informative

    y'know, when the FBI finally goes up against the Supreme Court

    Actually, all the unconstitutional crap is being done by the NSA. The FBI got warrents (over 120) through the FISA courts for every single aspect of the British plane bombers investigation that they participated in.

    which goes to prove that the NSA warrentless program is utterly unnecessary to stopping terrorism.

    --
    "But remember, most lynch mobs aren't this nice." (H.Simpson)
    -- Joe
  4. Re:Government Inefficiancy by diersing · · Score: 2, Informative
    Its not just governments, if you have a corporation dedicating a load of money towards some project the same will happen. Although the principles of project management, software development life cycles and security have matured, their adoption (of the processes) has not.

    So you end up with directors forcing managers, PMs and the like to adopt the formalized procedures and their unfamiliarity with the process leads to cost overruns and issues. So you outsource it and inevitably (every case I've personally seen anyway), you get cost overruns and finger pointing - outsource company says internal resource aren't responsive enough, internal resources say outsource company wants to dominate their time with daily conference calls and meetings that inevitable forces delays to the work being done. The bottom line, is no one ends up taking ownership of the work - they'd rather talk about it and run the billable hours up.

  5. Project Managers by Epeeist · · Score: 4, Informative

    Spit.

    The trouble with project managers (and security people) is that they have a checklist mentality.

    PM: Have you done this as yet.

    You: No, there is no need for it

    PM: But I need to get it checked off on my plan

    You: It shouldn't be on the plan in the first place

    PM: But it is on the plan, so I need to get it checked off. When are you going to do it.

    And so on.

  6. How is this news? by EnderGT · · Score: 5, Informative
    The FBI abandoned the VCF program in 2005. The replacement program, called Sentinel, is being led by Lockheed Martin. It is budgeted at $425 million, and won't be ready until 2009.

    Rereading the summary, the submitter has it wrong - "FBI's attempt to modernize their department has once again failed" implies that Sentinel has failed - which is definitely not the content of the article. Even the snippet quoted is about VCF having problems, not Sentinel.

  7. Re:Why not just hook it all up to a search engine? by cryfreedomlove · · Score: 3, Informative

    According to the Google Search Appliance FAQ, it will index PDF. Of course, that'll only work where PDF is comprised of text documents instead of images.

  8. Re:Sounds like the client was the primar one at fa by ChrisC1234 · · Score: 2, Informative

    You won't understand this until you've worked as a government contractor. When you are a contractor, the government employees are god (or at least that's what they think). In 90% of the projects that I have worked on for the government, it's the government employees who cause most of the problems. You are not given the authority to tell them "NO", and you must live with their idiotic decisions, even when you know that it will just cause more problems.

  9. Old News and No News? by Nerd_52637 · · Score: 3, Informative

    From the original post: the FBI's attempt to modernize their department has once again failed

    Failed once again? The article (you have to read the whole thing) says it's on track.

    The article is 90% about the Virtual Case File system ("built" by SAIC) and it's eventual demise in early 2005, almost 2 years ago. At the end, they discuss the FBI's replacement for VCF, saying:

    "Last year, FBI officials announced a replacement for VCF, named Sentinel, that is projected to cost $425 million and will not be fully operational until 2009. A temporary overlay version of the software, however, is planned for launch next year. The project's main contractor, Lockheed Martin Corp., will be paid $305 million and will be required to meet benchmarks as the project proceeds. FBI officials say Sentinel has survived three review sessions and is on budget and on schedule."


  10. Re:Safety First by Doc+Ruby · · Score: 2, Informative

    If you make $12 TRILLION a year, then you have the bigger problem. If you need to borrow more next year, and the year after that, and I can afford to let you slide if you just give me more control of the world you dominate, then you have the bigger problem.

    The US is no deadbeat - it's doesn't fail to pay its debts. It's among the best investments ever in the world. And its collateral is by far the best to seize.

    Besides, China cares nothing for shame. Its mafia government cares only for power. Power that Bush has handed it in unprecedented amounts. In exchange for lots of Chinese bribes to Bush's Republican Party

    --

    --
    make install -not war

  11. Re:Government Inefficiancy by GMontag · · Score: 4, Informative

    I am a defense contractor, Defense Financial Manager.

    Actually, the funds are someone's money. The Contracting Officers are legally "on-the-hook" for the things they sign for. If they authorize payment for something that was not delivered and the government does not get it's money back, then they are supposed to be liable for the money they released.

    If they continue working for the government a payment schedule is arranged and they have money deducted from their salary. If they get any other money from the government (ex:retirement) that is used toward the debt.

    The rules over here at DoD tend to be much more strict than at other agencies, contrary to what some in the media would lead you to believe.

    I hear that one of the problems now, with non DoD activities, is that there are not many prosecutions going on for that sort of thing. Also, the way these stories are written, there may not have been any wrongdoing at all (check my .sig) other than the exagerations by the reporter. It could be a case of a badly written contract that the government accepted, but if the terms for payment were met then nobody is on the hook for the money, but should be losing their job.

    In my case, since I am just a contractor and not a government officer, in this role, (in another position I am sometimes in uniform for the Reserves) I am never on the hook for the agency funds, but my customer is and if his error is due to my doing bad work then I am at risk of losing my job, which can happen with no notice.

  12. Not New Story sheds no new light by porslap · · Score: 5, Informative

    full disclosure: I wrote the "Who Killed the Virtual Case File" story for Spectrum, which ran last September.

    Here's some more food for thought about the "reporting" behind the FBI story:

    What's the news angle that warrants front page attention in the Post? That the Post reporters obtained the "unreleased" Aerospace report? Not news: the report was released to Spectrum at the end of April after nine months of litigating a Freedom of Information Act Request.

    All the Post reporters had to do was google "virtual case file" and voila! the story pops up as number 1, right there for them to rewrite!

    But say they are too lazy to bother googling. They just want the summary. The Spectrum article is the basis for the Wikipedia Entry on the Virtual Case File and the only external link. http://en.wikipedia.org/wiki/Virtual_Case_File

    The Spectrum article was the first and until the Post article, the only one to mention Matthew Patton, who was unearthed by dint of investigative reporting nowhere acknowledged in the Post article.

    The Post article purports to turn a spotlight on SAIC, in part by quoting David Kay, the Iraq weapons inspector, who was a former SAIC VP--but who had absolutely no firsthand knowledge of the VCF project.

    The Post article uncritcally takes FBI CIO Azmi's word that the follow up project Sentinel is on-budget and on-time, when other news outlets have recently reported about a growing sense within the FBI that this project is doomed to a fate similar to the VCF's.

  13. The Santa Monica bridge reconstruction by edremy · · Score: 5, Informative
    The I-10 bridge rebuild following the Northridge earthquake: details here

    This is how big government projects *should* be done. Hire a good contractor, set a minimum and then give bonuses for good performance and penalties for bad. Did the final tally cost a lot in bonuses? Yes. Was it worth it? Yes- they fixed a major problem in amazing time and did it correctly, plus they had a bunch of blue-collar folks make serious coin working triple time, all of which got plowed back into the local economy.

    You can argue it wasn't on budget due to the bonuses, but it was assumed from the beginning they'd be paying out. Since the daily economic loss to LA was higher than the daily bonus for finishing early, I'd argue it was actually under budget.

    --
    "Seven Deadly Sins? I thought it was a to-do list!"
  14. Re:Government Inefficiancy by Anonymous Coward · · Score: 1, Informative

    Yes.

    Lets say the Gov't agency shortlists to 3 vendors. These do contracts for many similar agencies. Each builds a full working system to choose from. The Gov't agency chooses the best. Now they have to pay the contractor 3x the cost of the system, just so the contractor can stay profitable considering the 2/3 of contracts it looses.

    So now you get a working system every time, but you spend 3 times as much as it should have cost in the first place.

    No, what the Gov't needs are two things:
    Use COTS/OSS products wherever possible
    Keep contracts to a digestible size

    It always seems that the bloated contracts are the ones that are most likely to fail. Far better to bid out in 10 pieces, and only have to replace the 30% that fail, than to have to replace the whole system.

  15. Re:Government Inefficiancy by Anonymous Coward · · Score: 1, Informative

    The FBI is a long way from getting an IT department thats worth anything.

    IT people who work there know you'll hit a cap very early on. The only way to make money in goverment IT is to be a private contractor. And the reality is, most good IT people expect a certain amount of money.

    And then what about those few IT people who work because they believe in the country? Give them a few months. Make a 1-line change in code and need to wait a month for approval because the layers are management and approvals are so thick, that illusion quickly fades. By the time you get approval you'll have noticed there are 2-3 more lines that have to be changed, now you need to get approvals all over again.

    They try to recover, again, with private goverment contractors where you can at least work.

    I've heard better things about military IT, where I at least know happy programmers there. They may not be paid the most, but they love who they work for (good managers), and they love the job.
    This of course varies, but military intelligence at least seems to treat people better.

    I've never met a happy programmer at the FBI.

  16. Re:Government Inefficiancy by Anonymous Coward · · Score: 1, Informative

    When corporate projects run into problems, they don't get shut down, the endpoints just get redefined. Once you kiddoos get out of mommy's basement and into the world; if you ever start experiencing how corporate life really works, instead of reading and thinking about how it ought to work, you'll know this.
    The extent of the government's role in the FBI project is just to accept or reject the system. The contractors who "manage" the project, and the contractors who do the tech work, are exactly the same ones who contract to AOL, or Sprint/Nextel, or any of the other private companies in this area. And, their projects are just as fucked up as anything happening in government. (Except maybe CSCs IRS project- but that's just because unmatched scale makes for unmatched fucked-upness.)

  17. Re:Government Inefficiancy by NialScorva · · Score: 2, Informative

    No, you won't.

    The software isn't tricky, it's the politics. The FBI is a feifdom of petty departmental tyrants and ancient practices. Several friends of mine have worked on the case file system in various incarnations, the problem is that the COTRs come in and define the requirements to be "exactly the same thing as this 25 year old main frame, but on a web page". One guy was having problems because his COTR was telling him that it had to be green text on a black background. That may have been an exageration, but based upon my experience it's quite belivable.

    I don't care how good of an engineer you are, you can't build a product for a customer can not or will not help you determine what their needs are.

  18. Re:Sometimes the problem is the specs. by bozendoka · · Score: 1, Informative

    Perhaps slightly OT, but related:
    I work for a large retailer that many here have no doubt frequented. A few years ago said employer outsourced ~95% of their IS department (thankfully I wasn't and still am not part of IS). The contract, as I understand, revolves around the company saving x amount of dollars, which they do admirably. It's simple really. It's so motherfscking impossible to get anything done through them that most departments either give up or work around IS entirely. Fewer projects == less money spent == savings!

    On the one hand I'm bitter, on the other hand I wouldn't have a job if they actually did their's.

    --
    "You will soon be more aware of your growing awareness." - My first recursive fortune cookie!
  19. Oblig. InfoWorld link to help out, too by PCM2 · · Score: 2, Informative

    I'd just like to point out that InfoWorld covered this story extensively last year.

    --
    Breakfast served all day!
  20. Re:Insanity by jafac · · Score: 2, Informative

    You could have a custom OS written in pure assembly for a quarter of that!

    Not from a CMMI-Level 5 organization (given all the paperwork, change management, formal testing, etc. that the Government Requires). - worse still - when you're talking about a DoD contract, add DISA STIG, and IA compliance, etc. etc. etc.

    --

    These are my friends, See how they glisten. See this one shine, how he smiles in the light.
  21. Re:Um, yeah, it's called "matching" by orac2 · · Score: 3, Informative

    Actually, Susan is my boss, but I'm going to assume you paid attention to where I indicated that in my original post, and are addressing your comments rhetorically. For the record Susan's been in journalism for decades, is a frequent judge for journalism awards, lectured at NYU's journalism school, etc, etc., and been the EIC of Spectrum for over six years. So drop the patronizing smarm. Finding your own angle on a story that's going around is one thing: failing to give adequate attribution is another, and is violation of, e.g., the Washington Post's ethics policy:

    Attribution of material from other newspapers and other media must be total. ... It is the policy of this newspaper to give credit to other publications that develop exclusive stories worthy of coverage by The Post.

    Certainly, for example, digging up Matthew Patton was an element of the VCF story that was exclusive to Spectrum's coverage, as Patton had not appeared in other media outlets before or since Spectrum's coverage until today.

    Even when publications are chasing the same story, when one publication gets something unique it is normal to see lines such as "As first reported in the New York Times..." etc in stories in other outlets. A similar attribution in passing in the text was all that would have been required: instead the only attempt at attribution by the Post article is buried in the credits list for the accompanying timeline graphic, where the "Institute of Electrical and Electronic Engineers" is credited as a source, which is a) insufficient and b) wrong (the source was "IEEE Spectrum Magazine". Crediting the IEEE is like crediting General Electric for information taken from a "Today Show" segment.)

    As a concrete example, let's look at the recent Sony-BMG DRM rootkit controversy. I did a story on that, interviewing many of the people involved, people who got interviewed by a lot of media outlets at the same time, but when I found a nugget that had been exclusivey reported by one other news outlet--a video of a DHS offcial talking to a local buisiness group about the issue--I gave credit where credit where was due. To the Washington Post in fact: "One party that cares is the U.S. Department of Homeland Security, which includes cybersecurity as part of its portfolio. On 10 November, as reported by the Washington Post, Stewart Baker, assistant secretary for homeland security, made a pointed reference to the Sony BMG protection system..." [Emphasis added]

    Speaking personally as someone who hires freelancers, and who's been a staff journalist and editor for somewhat more than a week myself, if your post is indicative of your grasp of the ethical standards of journalism, you can be sure this is one editor who wouldn't call on your abilities as a stringer, or anything else.

    --
    "Just once, I'd like to meet an alien menace that wasn't immune to bullets." -- The Brigadier, Dr. Who
  22. FBI Agent Writes Own Database, Becomes CIO by littlewink · · Score: 2, Informative
    While it sounds like a success story for FBI agent Depew, the IEEE Article about the VCF system underscores two gaping problems in the FBI's approach to IT:
    "Unfortunately, the FBI couldn't provide him with a database program that would help organize the information, so Depew wrote one himself.
    - Here we have an FBI agent with so little investigative work to do and so much time on his hands that he can write a DBMS! Why wasn't he prosecuting crimes and chasing the BGs? If indeed there was no software product available on the market, at worst he could have paid a programmer to do the job. This was a bored FBI agent who didn't want to do FBI work - he wanted to write software. But that's not his job! Depew should have been fired for writing a DBMS instead of doing investigative work. That's why specializations exists and that's why the FBI has software specialists.

    Since Depew was skilled enought to write his own PC -based DBMS, the FBI decided that he should be put in charge of a multi-million dollar project. This also was a SNAFU. Writing a PC program doesn't qualify you to manage a huge software project.

    One of the least known problems in law enforcement is keeping officers and agents focused on their work. They'd much rather take classes in programming, set up websites, build Access databases for the Captain, or in general do anything rather than get out on the street and do policing or legal work. The problem is, no matter what they do, they get the same pay. Policing or tracking down leads requires footwork and is physically demanding, so most veteran agents prefer a desk job.