Slashdot Mirror


DoD Ditches Open Source Medical Records System In $4.3B Contract

dmr001 writes: The US Department of Defense opted not to use the Department of Veterans Affairs' open source VistA electronic health record system in its project to overhaul its legacy systems, instead opting for a consortium of Cerner, Leidos and Accenture. The initial $4.3 billion implementation is expected to be the first part of a $9 billion dollar project. The Under Secretary for Acquisition stated they wanted a system with minimum modifications and interoperability with private sector systems, though much of what passes for inter-vendor operability in the marketplace is more aspirational than operable. The DoD aims to start implementation at 8 sites in the Pacific Northwest by the end of 2016, noting that "legacy systems are eating us alive in terms of support and maintenance," consuming 95% of the Military Health Systems IT budget.

97 of 186 comments (clear)

  1. A consortium of Cerner, Leidos and Accenture by Anonymous Coward · · Score: 5, Insightful

    Jesus Christ what a waste of money and to the worst possible people.

    1. Re:A consortium of Cerner, Leidos and Accenture by invictusvoyd · · Score: 3, Interesting

      We are deeply grateful to Source Forge for providing us with our place in cyberspace for this web site.

      http://worldvista.org/AboutVis...
      Agreed, but this sounds kinda weird too

    2. Re:A consortium of Cerner, Leidos and Accenture by Anonymous Coward · · Score: 1, Insightful

      Their only noncommercial option was the VA's VistA, requiring constant redevelopment to meet federal HITECH and ONC mandates and built entirely around a 1960s-era "key-value" database language called MUMPS. If you're a migraine enthusiast, enjoy night terrors, or otherwise are in the market for a drinking problem I recommend learning MUMPS.

      Here's a high-level summary of MUMPS:
      There is no database. Everything is stored in a multidimensional array. There are no ints or floats. Everything is stored and retrieved as a string. There are no reserved names because the entire language is context-based. There are no tables, tablespaces, or schemas. Everything in the global array is persistent and is saved directly to a disk. All program code is stored and executed within the global array alongside production data. Scared yet?

      P.S. MUMPS is also the underlying database technology of the other DoD front-runner, Epic Systems, widely regarded in the industry to be the most expensive EHR software ever developed. So all things considered, I think the DoD could have done worse.

    3. Re:A consortium of Cerner, Leidos and Accenture by Blaskowicz · · Score: 1

      There are no tables, tablespaces, or schemas. Everything in the global array is persistent and is saved directly to a disk.

      So the bloody thing is NoSQL and ready for the biggest revolution in computing soon hitting the market - persistent memory such as Memristor and 3D XPoint.

    4. Re:A consortium of Cerner, Leidos and Accenture by arglebargle_xiv · · Score: 5, Insightful

      A large-scale medical records system, a multibillion-dollar IT project, and companies like Accenture doing it, it's like combining herpes, syphillis, and gonorrhea and hoping you'll get a cure for cancer. Any of of those in isolation is pretty much pre-ordained to fail, and they're combing them all into one massive clusterfsck... why don't they just declare failure in advance and save the years of effort (and money).

    5. Re:A consortium of Cerner, Leidos and Accenture by Bonzoli · · Score: 1

      this is about acquisition officers future employment folks. Move along.

    6. Re:A consortium of Cerner, Leidos and Accenture by Anonymous Coward · · Score: 3, Informative

      You obviously don't know anything about MUMPS.

      I'll just discuss these specific points:
      - It *is* a database.
      - The "multidimensional array" is some sort of b-tree, the same kind of approach that is used by an RDBMS.
      - The language is dynamically typed ("no ints or floats"), which in practice means it's like working in Python or Perl. Internally, numerical data is stored in a numerical format for efficiency.
      - There's no need for reserved names, that's correct. Nonetheless, coding practices dictate that you don't name a variable "if" or something silly like that.
      - There is a schema.
      - Yes, a "global array" is a database object so of course it's stored to disk.
      - "All program code is stored and executed within the global array alongside production data." It's stored in a specific location (outside the database schema) that's dedicated to program code. These are basically stored procedures. I imagine every database you're familiar with also keeps its stores procedures in the database.

      I could educate you more thoroughly, but it would be a waste of my time.

    7. Re:A consortium of Cerner, Leidos and Accenture by Trax3001BBS · · Score: 1

      You obviously don't know anything about MUMPS.

      - There's no need for reserved names, that's correct. Nonetheless, coding practices dictate that you don't name a variable "if" or something silly like that.

      A variable of NULL would have no problems? As time goes on people are not aware.

      I've nothing against MUMPS and your write up informative but that one line caught me.

    8. Re:A consortium of Cerner, Leidos and Accenture by JustAnotherOldGuy · · Score: 2

      So...you're saying it's webscale? Thank god, because we were just about to do it the old-fashioned way so we would end up with a working product before the end of the century.

      --
      Just cruising through this digital world at 33 1/3 rpm...
    9. Re:A consortium of Cerner, Leidos and Accenture by K.+S.+Kyosuke · · Score: 1

      That a pointless question since if you have a brain, you're trying to avoid NULL in any form like the plague, so the variable name is free. ;)

      --
      Ezekiel 23:20
    10. Re:A consortium of Cerner, Leidos and Accenture by davester666 · · Score: 1

      It gave me spots all over, and I still lived!

      --
      Sleep your way to a whiter smile...date a dentist!
    11. Re:A consortium of Cerner, Leidos and Accenture by Anonymous Coward · · Score: 1

      It guarantees that the DoD managers of this project are guaranteed private sector jobs for life after they leave governmental "service"... Ain't the revolving door nice?

    12. Re:A consortium of Cerner, Leidos and Accenture by lsatenstein · · Score: 1

      Jesus Christ what a waste of money and to the worst possible people.

      Think of the money you can earn, working to fix the consortium's system and then a contract to convert their systemto the open source version.

      Wow, A cast of hundreds with salaries to three times the norm.

      --
      Leslie Satenstein Montreal Quebec Canada
    13. Re:A consortium of Cerner, Leidos and Accenture by terjeber · · Score: 1

      Isn't it more like getting infected with Ebola to cure the common cold. Sure, it kills the cold. And the host.

  2. 300 million for the hardware/software/salaries by Anonymous Coward · · Score: 1

    4 billion leftover for all the people in charge.

  3. Trading one for the other by grilled-cheese · · Score: 4, Insightful

    It's interesting how they see integrating legacy systems any differently integrating just as many differently implemented commercial record systems. The data integrators will make the same money either way. By abandoning the open-source solution, you're just losing the possibility others might benefit from the work. Likewise, I'm curious how much those 3 vendors have lobbied in Washington DC.

    1. Re:Trading one for the other by Anonymous Coward · · Score: 1

      I know people working on the veterans affairs stuff... if they made this choice, they didn't make it lightly...

      In particular their legacy system was a crap ton of post-it notes stuck to computer screens.

      The Obama administration has gotten a lot of help from Google's SRE groups moving in to help fix the government with everything they learned at Google. These are not bureaucratic choices being made, they're Google SRE choices...

    2. Re:Trading one for the other by drooling-dog · · Score: 2

      I have no knowledge of the particulars in this case, but lobbying isn't even really necessary. It's often just the revolving door: The procurement people on the government side now have very lucrative careers in the private sector to look forward to, and that is something you can never get by going with the open source solution. But who knows, maybe this time they did make the call purely on its technical merits.

    3. Re:Trading one for the other by Anonymous Coward · · Score: 2, Insightful

      The open source solution was not a good one in this case. You can have religious wars all you want about which language is best, but when striving for worst, MUMPS is a real contender. Please don't copy the database structure of VISTA. It's utterly useless for data integrity -- imagine a database where every field is a string. There are no numeric fields...and what happens? Oh.

      The biggest problem is that the DoD lacks the organizational and technical skills at higher levels to use anything other than a defacto standard. They need something akin to RFC's, but that is not the military way.

      The $9B price tag is actually reasonable for a good implementation and rollout. We know history too well: no one will see anything from the project for 10 years, and it will be rolled out on top of a lackluster Oracle database and dozens of middleware pieces that will ensure the system remains more expensive than current operation costs forever. And it will probably be less functional that what it replaces.

      It's a shame that such big consumer lacks the power to make a difference in the market by developing and promoting good standards.

      Yes, I'm bitching. I've been in the system long enough to lose hope.

    4. Re:Trading one for the other by phantomfive · · Score: 5, Insightful

      Regardless of whether they are starting with open source software, or closed source software........if I ever paid $4.3 billion for some software, I guarantee I would be getting the source for it. If the government pays that much for a system, one of the requirements should be that it ends up open source.

      --
      "First they came for the slanderers and i said nothing."
    5. Re:Trading one for the other by Fire_Wraith · · Score: 2

      It's not necessarily even corrupt, for that matter. It's about having the personal relationships, and moreover, knowing how the labyrinthine mess that is the DoD Acquisition process works. The rules are intended to keep it fair, but at the same time, also wind up pricing a lot of the inexperienced sorts out of the process simply because you have to know what language to use, how to structure it, etc.

    6. Re:Trading one for the other by gtall · · Score: 1

      And if DoD decided to do it inhouse, the libertards would be hanging from the lampposts crying the blues about big government.

    7. Re:Trading one for the other by j-beda · · Score: 1

      Regardless of whether they are starting with open source software, or closed source software........if I ever paid $4.3 billion for some software, I guarantee I would be getting the source for it. If the government pays that much for a system, one of the requirements should be that it ends up open source.

      At least.

    8. Re:Trading one for the other by CaptQuark · · Score: 1

      Why? The government is buying a service, not a software package.

      Imagine if the government contracted Gmail to provide unclassified email for the entire DoD for 10 years for 4.3 billion. (2 million active and civilian accounts x 10 years = $215 a year per account.) Why would DoD want the source code for Gmail just because they are paying for the service? DoD isn't going to turn it into open source and they certainly aren't going to try to maintain a separate Gmail system. Google isn't going to bid $215 per account if they have to release Gmail into open source. In this fictitious example, DoD is paying Gmail for a proven email system and the personnel to keep it running, up to date, and secure.

      Similarly, if GSA contracted Ford to supply, maintain, and refresh every government vehicle, do you think GSA would demand the design and engineering specs for every vehicle it leased?

      In most cases the government is paying for a service because it doesn't WANT to own or maintain something that is already a mature civilian industry.

      --

    9. Re:Trading one for the other by phantomfive · · Score: 1

      When things go right, you don't need the source code, but when things go wrong, you'll wish you had it.

      Also, as to your gmail example, if the DoD plans on storing medical records in someone's crappy 'cloud,' they are giving themselves even more problems. You want to be able to control that data.

      --
      "First they came for the slanderers and i said nothing."
    10. Re:Trading one for the other by CaptQuark · · Score: 1

      When things go right, you don't need the source code, but when things go wrong, you'll wish you had it.

      Again, Why?

      Do you think DoD is going to tell the company how to rewrite the code to fix it? When you write the contract you put in performance specifications (remember five 9s?) and penalties for non-compliance. You don't spend months reviewing the vendor's software code to find edge cases that cause problems. DoD is paying them for their expertise, manpower, and infrastructure that DoD doesn't have to maintain.

      --

    11. Re:Trading one for the other by phantomfive · · Score: 1

      In their case, it will allow them to switch vendors more easily.
      Also, programmers write better code when they know people will be looking at it. The open source method for motivating better programming.

      --
      "First they came for the slanderers and i said nothing."
    12. Re:Trading one for the other by K.+S.+Kyosuke · · Score: 1

      It's utterly useless for data integrity -- imagine a database where every field is a string. There are no numeric fields...and what happens? Oh.

      Somehow I don't think a computer system should have a problem with this. Ultimately, in every computer system, every field in a database is an 8-bit string. On some kind of disk. ;) Actual lack of checks is what would give you problems.

      --
      Ezekiel 23:20
    13. Re:Trading one for the other by nabsltd · · Score: 1

      In this fictitious example, DoD is paying Gmail for a proven email system and the personnel to keep it running, up to date, and secure.

      The other huge difference in your example is that DoD would merely be paying for something already built and functional.

      TFA is about three companies building a system from scratch. Even if they use something that mostly already exists, it will be highly customized for DoD, to the point that it will be different from anything they have already done, and nowhere near "proven" or "tested". Since the history of those three companies shows they want nothing more than keeping their clients locked in as long as possible, it's not a recipe for long-term success.

    14. Re:Trading one for the other by FranTaylor · · Score: 1

      Also, programmers write better code when they know people will be looking at it. The open source method for motivating better programming.

      citation required

    15. Re:Trading one for the other by phantomfive · · Score: 1

      Do you think it's not true? Prima Facie it seems obvious, based on my own experience, and watching others.

      --
      "First they came for the slanderers and i said nothing."
    16. Re:Trading one for the other by CaptQuark · · Score: 1

      [comment redacted]
      The first rule of SIPRNet is you don't talk about SIPRNet. -_^

      --

  4. follow the money by Lead+Butthead · · Score: 4, Interesting

    follow the money and the answer is in front of you.

    --
    ELOI, ELOI, LAMA SABACHTHANI!?
  5. $9 billion dollar project? by Anonymous Coward · · Score: 1

    Try paper, it still works.

    1. Re: $9 billion dollar project? by snowgirl · · Score: 2

      Paper doesn't scale to the level required. Trust me, I've attended presentations from ex-Googlers on the topic.

      Especially, when the paper weighs so much that it started deforming a building...

      --
      WARNING! This girl exceeds the MAXIMUM SAFE standards established by the FDA for BRATTINESS
    2. Re: $9 billion dollar project? by bigfinger76 · · Score: 1

      In NC, the "storage floor" for all the backlogged records is sagging, threatening to collapse. Paper is not going to work.

    3. Re: $9 billion dollar project? by Blaskowicz · · Score: 1

      Maybe it's in the basement and menacing the soil's integrity ; earth itself can collapse under the weight of a few years of DoD paperwork.

  6. $4.3 billion == guaranteed failure. by serviscope_minor · · Score: 5, Insightful

    As far as I can tell, any IT project costing a billion or more is 100% guaranteed to fail.

    Also, it sounds like they decided to source IT from Lufier, Mephistopheles and Satan, which incidentally also guarantees it to fail.

    --
    SJW n. One who posts facts.
    1. Re:$4.3 billion == guaranteed failure. by wonkavader · · Score: 4, Insightful

      Tom DeMarco talks about the air traffic control software project in one of his books. The description of the hopeless situation in that case supports your idea.

      I think when you have a lot of people's butts on the line and so failure is not an option but stagnation IS, what we would perceive as failure is almost certainly coming. You can retire without any fallout so long as you make sure nothing happens for 15 years. It's easy to do: Just make the specs vague, self-contradictory, and long. Very, very, long.

      The project won't fail, but it won't succeed either. And you're safe, which is all that matters.

      They would do much better to set up a few small teams and have them compete to build something with enough in common so one can be replaced by the other. And starting with the open source base would make sense there.

    2. Re:$4.3 billion == guaranteed failure. by dinfinity · · Score: 5, Insightful

      As far as I can tell, any IT project costing a billion or more is 100% guaranteed to fail.

      No kidding.

      If you pay everybody $200 000 per year, that equates to 21 500 man-years (!) of work. I don't know what kind of problems in record keeping they're going to solve, but for that kind of money it'd better involve employees doing that in gold plated jets flown by an artificially engineered unicorn that continually snorts prime-grade cocaine.

    3. Re:$4.3 billion == guaranteed failure. by Rich0 · · Score: 1

      Just make the specs vague, self-contradictory, and long. Very, very, long.

      When you get to software on this scale just writing the specs is an insanely difficult problem. Most big systems fail due to poor requirements, and I think that is because good requirements are a LOT harder to get right than people appreciate, especially for waterfall-style projects that are always the favorites of big RFP projects like these.

    4. Re:$4.3 billion == guaranteed failure. by superwiz · · Score: 1

      Good luck finding lawyers competent with the government's requisition procedure who work for as little as 200k a year.

      --
      Any guest worker system is indistinguishable from indentured servitude.
    5. Re:$4.3 billion == guaranteed failure. by K.+S.+Kyosuke · · Score: 1

      These days, you could almost fly to the Moon for that kind of money.

      --
      Ezekiel 23:20
    6. Re:$4.3 billion == guaranteed failure. by dinfinity · · Score: 1

      Screw your implicit apologistic bullshit.
      You could hire 100 lawyers and pay them $1 000 000 yearly for 10 years and still have 3.3 billion dollars left.

      This is the industry suckering the government out of their cash and you know it.

    7. Re:$4.3 billion == guaranteed failure. by superwiz · · Score: 1

      Good luck getting lawyers who could get a top government contract for $1000,000 a year. This is what top divorce lawyers will bill nowadays. Top Fed-contract requisition lawyers are probably asking at least 10x as much. But they could be asking 100x or a 1000x as much. Because they'd still be worth it if they return this much on their effort.

      --
      Any guest worker system is indistinguishable from indentured servitude.
    8. Re:$4.3 billion == guaranteed failure. by dinfinity · · Score: 1

      You are veritably ridiculous.
      You are arguing that there are lawyers that could ask 1 billion dollars yearly. 'Because they'd still be worth it'.

    9. Re:$4.3 billion == guaranteed failure. by superwiz · · Score: 1

      Yes, I am definitely arguing that someone who can bring in a $4.5 billion contract which takes less than a billion to fulfill is, in fact, someone who deserves their 30% billion dollar paycheck. If someone threw $3.5 billion in free money my way, yeah, I'd say they deserve their $1 billion finder's fee.

      --
      Any guest worker system is indistinguishable from indentured servitude.
    10. Re:$4.3 billion == guaranteed failure. by dinfinity · · Score: 1

      That is not how that works. At all.
      You have at best a primary school level understanding of capitalism and economics in general.

    11. Re:$4.3 billion == guaranteed failure. by superwiz · · Score: 1

      oh. you are one of those. ok. i guess i should have known better than to argue with a millennial.

      --
      Any guest worker system is indistinguishable from indentured servitude.
  7. Accenture? by Anonymous Coward · · Score: 3, Insightful

    Project already failed.

  8. UK NHS by martin · · Score: 3, Informative

    Sounds alot like the disasterous Nhs epr systems. After 20 years weve had progress but the number suppliers is down 1 and a major cash sinkhole

    1. Re:UK NHS by dmr001 · · Score: 2

      I thought about the NHS program when posting this, described as "the biggest IT failure ever seen". After £10 billion+ was spent, Her Majesty's government largely abandoned the effort, though the linked article notes Computer Sciences Corporation declaring victory as 3 of 220 NHS trusts managed to use portions of the system. I first heard this story a couple of years ago on a shuttle bus to the headquarters of a large privately held EMR vendor in Wisconsin, when I noticed the accents around me weren't American (like me). I was sitting amongst a group of friendly pharmacists from Oxfordshire. They were going to adopt this proprietary system for their NHS trust (ignoring, I suppose, the large chunk of it that dealt with billing).

      Besides the air of defeat of all those pounds sterling going down a lot of oddly designed British toilets, they had given up on the idea of interoperability with the systems of other NHS trusts adopting different systems from other proprietary vendors. Back in the US, we have all kinds of government prodding to promote interoperability and many self-congratulatory health IT standards organizations that have national meetings in sunny placed. But, the farthest we've got with inter-vendor communication in my medical office after 3 years of promises and finger-pointing is faxing documents to an image server from the speciality clinic 100 feet away into inscrutably named files. Then, I can hand transcribe the important bits by hand about my patient's heart conditions and colon tumors in order to have a hope of retrieving that information again when I need it.

    2. Re:UK NHS by TeknoHog · · Score: 1

      Sounds like EPR needs CPR.

      --
      Escher was the first MC and Giger invented the HR department.
    3. Re:UK NHS by Dr_Barnowl · · Score: 2

      We do have interoperability standards : I used to work for the department that specifies them.

      They just don't follow them. Case in point : the meta-standard (HL7 v3) that we used for our messaging had a mechanism for not just sending NULL values, but also sending a reason why they were null. (e.g. - the value wasn't measured, etc). The vendor had no truck with that, and was using magic numbers instead (e.g. baby weights of 9999g which is outside the realms of sanity for a newborn). I was tasked with revamping one of the messages. I specified that the proper NULL flavours get used and ditch the magic numbers.

      The vendor at this point rolled out the "full system test" clause in their contract, whereby they could charge £N * 10^6 to perform a full system test, because we'd changed the behaviour of one field. They got their way and kept their magic numbers. Other systems expecting messages that met with the conventions of the overall meta-standard for data now have an additional development cost to cope with those magic numbers.

      This is the reason for the focus on interoperability over just having standard data structures - it lets vendors continue to use their own proprietary data schemes, and raises a barrier to new participants in the market, not only do you have to implement all the standard interfaces to interoperate, but you probably also have to design in a "quirks" layer to cope with each vendors *special* variations.

  9. Sometimes updating legacy systems is a real pain. by Anonymous Coward · · Score: 1

    The hardest part of legacy is that sometimes the software just does what it does, and no one has all the official rules to recode it. So sometimes they task unwitting coders with learning some random programming language people don't use anymore, and most people didn't even use back in the day. They task them to do that to find out what rules they're operating under by reading the spaghettiest of spaghetti code. That is by no means a fun job especially when you're talking about governmental designed rules. I can't explain just how tough this job is if they don't even know their own operating procedures. If they do know their own operating procedures, then just update the database, write the fresh code in a modern language and send it out the door.

  10. As an Accenture stock holder by Anonymous Coward · · Score: 1

    As an Accenture stock holder I think this is definitely a step in the right direction.

  11. $4.3 billion by lkcl · · Score: 4, Insightful

    wow fuck. imagine how much advancement in software libre could be had for $4.3 billion if the contract had been awared. hell, even 1% of that would make a big fucking difference. someone - such as the gnumed developers to take even one random example - could, with help, have developed a medical records system for ohhh i dunno... the U.S. Dept of Defense, with that kind of money. just to take a random example, y'ken.

    1. Re:$4.3 billion by gtall · · Score: 2

      You have no idea of scale.

    2. Re:$4.3 billion by terjeber · · Score: 1

      Really? Put that much money into open source and you'll have one fork for each $20 put in. Then the project is going to be complete in the mythical Year of the Linux Desktop (TM)

  12. Speakings of the OSS in question by ADRA · · Score: 1

    They use MUMPS. I know its all supported by some people and I know the flames are coming, but really? MUMPS. I'd say integration could very well be considered a pain point in the language. In my cursory investigation, the recommended integration for other languages / technology chains is a Node.js based web services adapter... oh well.

    --
    Bye!
    1. Re: Speakings of the OSS in question by Dr_Barnowl · · Score: 1

      Tyson, is that you??

  13. Part of the problem by satsuke · · Score: 3, Insightful

    That is part of the problem .. trying to design a drop in replacement that replicates the current functionality and interoperability with other systems.

    With government especially, you have lists of exceptions and custom one-off code to get something working, that it becomes impracticable to replace it without an equal or additional number of exceptions.

    It's the kind of system that benefits from a "flush it all away" mentality of defining new standards and sticking to them.

  14. Eating them alive by Todd+Knarr · · Score: 1

    Well, it'll solve the problem of the system eating them alive in terms of maintenance and support. Now it'll be eating them alive in terms of development costs instead.

    No, wait, they'll need to keep the legacy systems running until the new ones are running, so it'll still be eating them alive in terms of maintenance and support too.

  15. Accenture? by msobkow · · Score: 3, Informative

    Accenture? Better double that initial estimate to $18 billion, and count on it rising further. :(

    --
    I do not fail; I succeed at finding out what does not work.
  16. Re:Good response by Anne+Thwacks · · Score: 1
    I should have thought that it was obvious to everyone that, for infrastructure, open source is the ONLY choice.

    Iit is the only way to guarantee it can be fixed.

    I offer MS Word in evidence.

    --
    Sent from my ASR33 using ASCII
  17. not enough bribes by OSS by dltaylor · · Score: 2

    US Government purchasing "works" by the payment of bribes. Usually, these are not simply cash payments, but the opportunity for lucrative "consulting contracts" at the providing companies for senior Penagon and Civil Service officials after leaving government "service". I've seen it enough to know that saving money, at equal or better performance, will not get a government contract. Maybe, if enough congresscritters and/or senaturds are bribed with campaign contributions and/or honorariums, they'll push a deal one way or the other, but that rarely has anything to do with saving purchasing costs.

    1. Re:not enough bribes by OSS by eulernet · · Score: 1

      Exactly !

      In fact, people signing large contracts always expect large kickbacks in return, at least 15% of it.
      This will probably finance the next presidential campaign.

  18. gnu project? really? by Anonymous Coward · · Score: 4, Insightful

    And who, in the gnu community, is going to take on the responsibility for all the enterprise scale stuff that needs to be done. I can see lots of folks wanting to scratch their particular itch by coding up some piece, but who's going to do the architecture design, ride herd on the developers, etc.; make sure that the documentation gets done and is usable and readable (because, ya know, all those packages out on github and sourceforge are ever so well documented)..

    I mean responsibility as in "be willing to stand up in front of Congress and explain your progress or lack thereof". I don't see a Linus or Theo or Eric or, gods forbid, Richard, filling that role.

  19. Re:Accenture by chipschap · · Score: 2

    OK, years late, most promised feature not done/working, Budget 3x+ over and you will move to a proprietary commercial product with 100% lock in.
    Sweet, yes would be fun to see who's back accounts/family members got hired to land this future failure in motion ;)

    You left out the part where no senior executive service (SES) people get fired but middle-level people who actually tried to make it work get the blame. Meanwhile the Beltway Bandits make off with millions and hire the SES people for fat salaries as a reward for sending all the money their way.

  20. Why? by Anonymous Coward · · Score: 1

    ... part of a $9 billion dollar project ...

    We all know the US department of defense demands a fixed-cost project then signs an open-cost contract. The usual excuse being the contractor must build the software from scratch. Since this project copies the contractor's existing system, the government can easily demand a fixed-cost contract. Like that's going to happen!

    ... a system with minimum modifications ...

    So they're paying $9 billion for something that's already been built. It sounds overpriced already.

    ... interoperability with private sector systems ...

    The US DOD is always saying they need a do-everything system that prevents data fiefdoms. It's why they're always re-inventing the wheel, so copying a non-standard, legacy system is the worst possible 'upgrade' and it throws away all the work done on VistA.

    ... is more aspirational than operable ...

    Maybe this is an attempt to not re-invent the wheel: By buying whatever is available instead of building a do-everything system from rainbows and unicorns. But the VistA system is also available, so no, it's a clever way to pay kickbacks.

    ... eating us alive in terms of support and maintenance ...

    So the answer is another legacy system? I'll bet the hardware for this new system will be 2 years-old. Part of that is the slow procurement system and part is the mIl-spec approval program that all DOD purchases must satisfy.

  21. VistA is a nightmare by Gravis+Zero · · Score: 5, Interesting

    i was interested in VistA and what all the fuss was about, so i decided to check it out. turns out the backend is nightmare code that would would swear was machine generated. after some investigation i found out it's MUMPS (Massachusetts General Hospital Utility Multi-Programming System) code. a lot of useful stuff started way back in the 1970s... but MUMPS is a 1960s nightmare come to life.

    think i'm exagerating? here's a module from VistA's code which was apparently updated in 1989.

    DENTA1 ;ISC2/SAW,HAG-DENTAL TREATMENT DATA SERVICE REPORTS ; 1/10/89 11:08 AM ; ;;1.2;DENTAL;**24**;JAN 26, 1989
      D:'$D(DT) DT^DICRW S %O="OPT",U="^",S=";",O=$T(@(%O)),DENTV=$$VERSION^XPDUTL("DENT") I $D(^DOPT($P(O,S,5),"VERSION")),(DENTV=^DOPT($P(O,S,5),"VERSION")) G IN
      K ^DOPT($P(O,S,5))
      F I=1:1 Q:$T(@(%O)+I)="" S ^DOPT($P(O,S,5),I,0)=$P($T(@(%O)+I),S,3),^DOPT($P(O,S,5),"B",$P($P($T(@(%O)+I),S,3),"^",1),I)=""
      S K=I-1,^DOPT($P(O,S,5),0)=$P(O,S,4)_U_1_U_K_U_K K I,K,X S ^DOPT($P(O,S,5),"VERSION")=DENTV
    IN I $P(O,S,6)'="" D @($P(O,S,6))
    PR S O=$T(@(%O)),S=";" S IOP=$I D ^%ZIS W:IOST'["PK-" @IOF K IOP
      I $P(O,S,7)'="" D @($P(O,S,7))
      E W !!,$P(O,S,3),":",!,$$VERSION^XPDUTL("DENT")," ",$P($T(+1),S,1),!!,$P(O,S,4),"S:",!
      F J=1:1 Q:'$D(^DOPT($P(O,S,5),J,0)) S K=$S(J0 S Z2=Z1
      G:Z3=0 W I Z3>1 S DIC="^DENT(225,",DIC(0)="AEMNQ",DIC("A")="Select STATION.DIVISION: " S:$D(DENTSTA) DIC("B")=$S(DENTSTA[" ":+DENTSTA,1:DENTSTA) D ^DIC Q:Y

    --
    Anons need not reply. Questions end with a question mark.
    1. Re:VistA is a nightmare by Dutch+Gun · · Score: 2

      Wow... I mean... wow. This is the definition of a "write-only" language. I'm pretty sure you'd need external documentation just describing what these routines do, because it sure as hell looks like you're not going to derive it from this encryption disguised as source code. I thought perhaps you had chosen some particularly horrible section, like maybe it was a data definition of some sort. Nope, after sifting through a bunch of code, it pretty much all looked like that. My brain hurts just trying to parse and make sense of some of that code.

      Is it any wonder that, with a language like this, it can't be easily extended and upgraded to meet demands? Yeah, okay, it's understandable now why they're tossing the whole thing. Open source or not, I can't imagine there are many people today who are able to extend or even maintain this monstrosity in perpetuity.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    2. Re:VistA is a nightmare by Blaskowicz · · Score: 1

      If you expand the crazy one-letter commands into their full length and color them, maybe it's a bit less terrible.

    3. Re:VistA is a nightmare by Dutch+Gun · · Score: 1

      Perhaps, but that's not what they're working with, since this is the original source. This language and it's horrible style was obviously invented in a day when the size of your variables and source files was actually a real consideration, and machine efficiency was prioritized over programmer efficiency. It looks like assembly language for distributed databases.

      That's precisely the opposite of what most large-scale enterprise systems need nowadays. Computers are incredibly powerful and cheap and programmers are very costly, so modern languages should (and do) reflect that reality.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    4. Re:VistA is a nightmare by Anonymous Coward · · Score: 1

      Cerner does not use M for anything. Cerner developed an in-house language called CCL (Cerner Command Language) which is standard SQL with additional functions built in, mostly for writing, saving and running scripts. All custom functions written for Cerner are typically done in CCL. The backend server code is almost entirely C++ on the backend, with a few pieces done in Java. Frontend code is roughly the same mix.

      As for databases, it's Oracle RDBMS and MySQL.

      Source: Former Cerner software engineer.

    5. Re:VistA is a nightmare by bugs2squash · · Score: 1

      OK, so build a new system, a superset of MUMPS that adds the needed functionality but continues to support the legacy code as well as some newer, better syntax.

      That way you can transfer in all the old crap and replace it bit by bit over some time. I'm sure I've even seen technology that analyzes executables and re-factors them - perhaps similar techniques could be applied to the old MUMPS source. Sure it's hard, but with $9B and several years I'm sure a handful of CS guys at a university department could achieve something truly exceptional.

      --
      Nullius in verba
    6. Re:VistA is a nightmare by Eravnrekaree · · Score: 1

      That code is written like every byte costed $500. Don't let this reflect badly on M. The problem is they used one character variables and no formatting. You can wrote worse code in C. M is perfectly readable if you use good variable names and formatting.

    7. Re:VistA is a nightmare by Eravnrekaree · · Score: 1

      It would have been best to hire some in house programmers , certainly could have been done with some portion of the few billion, and have them write a new system, which could then be made public like VISTA. This way the government owns it, instead, there will be an opaque system and the government will be held captive to extortion from providers.

    8. Re:VistA is a nightmare by JustAnotherOldGuy · · Score: 1

      That code gave me brain cancer.

      --
      Just cruising through this digital world at 33 1/3 rpm...
    9. Re: VistA is a nightmare by Anonymous Coward · · Score: 1

      Anyone able to use a touring complete language in a productive way can learn basic M in a day or two.

      Expand the commands from their one-letter form and a programmer most likely can read the code within the time it takes to read up on the language: what the 26 commands do, how to apply conditionals to individual statements and the format to chain commands (use of syntactic whitespace) together inside a routine.

      Using it to the fullest will take longer because one needs to grasp the elegance of the design to realize why constructs commonly seen in other languages seem to be missing in M.

    10. Re:VistA is a nightmare by serviscope_minor · · Score: 1

      You can write MUMPS like that, sure, but then you can either write a good kernel in C or win the IOCCC. If one sticks to modern conventions such as using full length keywords, not refedining keywords and actually indenting (pretty much all things you can also do in C, even if redefining keywords is technically illegal, I've never seen it not work), then the code as far as I can tell looks fine.

      But even if all the code looks like that, for $9,000,000,000, you could reformat then refactor the entire lot, then write a new mumps compiler which enforced some coding standards and you'd still save more than $8,000,000,000.

      --
      SJW n. One who posts facts.
    11. Re:VistA is a nightmare by K.+S.+Kyosuke · · Score: 1

      He essentially suggested the use of automated techniques. From what I can tell, such automated techniques are being applied in practice. I recall a project that used a Smalltalk-based code analysis platform to refactor COBOL code. Are you saying that this approach doesn't save time and man-hours?

      --
      Ezekiel 23:20
    12. Re:VistA is a nightmare by K.+S.+Kyosuke · · Score: 1

      The one interesting thing here is that if you rewrite the execution environment, without creating a new language, you should be able to use the old code as tests for the functionality of your new code. You need some way to set up and tear down state for the snippets you're going to be executing, so the old execution environment that ran the system in production probably won't be suitable. Your best hope would probably be starting with a new "implementation" that can parse the code and either interpret it, or compile it, while recording the things that may be of interest to you (for example, if the old language was dynamic, and you need type information either for the purpose of documentation, or to generate static types for the new implementation, you can recover it by tracing).

      You can't really use old code as a spec if you lack the necessary tools. Or, perhaps you could, but you'd be doing in your head what a computer should be able to do for you automatically. Expect working very long hours if you're so intent on doing a compiler's and code analyzer's work yourself. ;-)

      --
      Ezekiel 23:20
    13. Re:VistA is a nightmare by arglebargle_xiv · · Score: 1

      E W !!

      Couldn't have put it better myself.

    14. Re: VistA is a nightmare by bill_mcgonigle · · Score: 1

      Anyone able to use a touring complete language in a productive way can learn basic M in a day or two.

      Yeah, the hospital I used to work at would take tech school kids and teach them M in six months, and then have a competent programmer. It's not work I wanted to do, but it's a real job. Jesus, hasn't anybody here done assembly?

      Using it to the fullest will take longer because one needs to grasp the elegance of the design to realize why constructs commonly seen in other languages seem to be missing in M.

      The best thing about a Jim Jones joke is the killer punchline.

      Seriously, I did Perl vs. M in the '90's with co-workers. Don't even. It was possibly a good argument in 1981.

      M persists because managers are afraid of making big changes and those big changes cost more up front than staying with the existing junk. Also they have a cartelized industry to draw resources from, so cost-competitiveness is not really a factor. Third-world countries are running their medical-records systems on "low-end" modern stacks because that's all they can afford.

      --
      My God, it's Full of Source!
      OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
    15. Re:VistA is a nightmare by phantomfive · · Score: 1

      That is beautiful. I haven't seen code like that for a long time.

      Of course, if you look at the individual page of code for a while, you'll decipher it easily enough. Whether it's manageable or not depends on whether the overall structure is reasonable or not.

      --
      "First they came for the slanderers and i said nothing."
  22. Queensland balls up by bigtreeman · · Score: 1

    Of course it's costing 95% for support and maintenance, there are no licensing or purchase costs. Would that last 5% be hardware costs, management costs, biscuits for morning tea ?

    When is a commercial EHR system not proprietary and when is an open source electronic health record system proprietary ?

    Governments all over the world are busting to give OUR money away to big corporations !!!

    Projections of $11B over 18 years, who's a betting man ? will it double or triple over the 18 years ???

    The health department system implemented in Queensland, Australia was a total balls up. IBM ?

    --
    Go well
  23. Re:4.3 billion is not enough. by tomhath · · Score: 1

    Not quite. What they want is a system which has interfaces available that any other vendor (open source or not) can use. Cerner has been moving in that direction; of the vendors out there they are probably the least bad choice.

  24. Has anybody checked out the VistA help doc !!! by sandGorgons · · Score: 1

    Here Jeez, it seems they use the GT.M embedded database and a GUI that connects directly to this database... kind of like PGAdmin3. They use the MUMPS scripting language to build logic into the database (kind of like stored procedures). the deployment doc on Linux is a mess. Wow!

  25. Re:Good response by Dutch+Gun · · Score: 1

    I offer MS Word in evidence.

    You'd be better off choosing as an example something other than a piece of software that a significant percentage of businesses on the planet pay for and use with no significant problems whatsoever. It could be argued that MS Word may be one of the most successful pieces of software in history. There are plenty of examples of terrible closed-source, proprietary software. Why not use one of those?

    --
    Irony: Agile development has too much intertia to be abandoned now.
  26. Re:Sometimes updating legacy systems is a real pai by Curlsman · · Score: 1

    "Space is hard" and this is harder, because it's about ourselves.
    This has no chance until someone like Pres. Kennedy chooses to do it "because it is hard" to do.

    I think this, updating legacy system (something I do), is close to the second problem, which is that no one person or team has a grasp on how it works now, and what ever is put in place will be in the exact the same situation: no person knows it all. The only way I've seen this addressed is that after the Big Problem is cut into the Smaller Problems, the people involved with each Small Problem have to be to only ones who understand it: it must be accepted that there will never ever be a global "this is how it works" person or team.

    Then the first problem can be addressed: interoperability. Each Small Problem can then announce (or publish) how they solve their problems in their area of expertise. Then tell all the other vendors that they will interoperate with them in that way. This is not the way this has been stated so far, where the new system will interoperate to all the others, which won't happen for the simple fact that all the other vendors don't want to, its not in their self interest.

    The third problem is inherent in our species: we change, so no static model can be used to organize the data about us. The next versions of HIV, Ebola, anti-Vaxers, are all changes that will strain any extensibility, but most especially a government/military/project planning approach that can only deal with statically defined, long term goals. In the 18 years of this project they will always play catchup.

    So, yes, centrally defining EMR is going to fail, absolutely (or the Soviets would still be a power...).
    My only poor suggestion is to find a way to give each medical specialty, including ones yet thought of, a way to define their own representations of their expertise and publish it so those working in that field can interoperate with each other. Something like a distributed/grid processing system that includes distributed people being involved.

    Google searching still requires that you know that you have the results you want when you see it, where a library is organized to begin with, so Library Science has been trying to do this for a centuries and it might be worth studying to understand some of the problems.

  27. Not all Open source is good. by bloodhawk · · Score: 1

    VistA is a shit legacy system, so it seems DoD made Half a right choice. However given the companies they got involved they will just end up with something almost as bad, if not worse.

    1. Re:Not all Open source is good. by serviscope_minor · · Score: 1

      VistA is a shit legacy system,

      From Stroustrup's FAQ:

      What is "legacy code"?

      "Legacy code" is a term often used derogatorily to characterize code that is written in a language or style that (1) the speaker/writer consider outdated and/or (2) is competing with something sold/promoted by the speaker/writer. "Legacy code" often differs from its suggested alternative by actually working and scaling.

      The VA system probably works. 50 bucks says the $9,000,000,000 system will never work.

      --
      SJW n. One who posts facts.
    2. Re:Not all Open source is good. by dmr001 · · Score: 1

      Guess which large-scale EMR physicians prefer above all others? That would be VistA. I've heard the same from colleagues, and found it reasonably sensible back when I rotated through the local VA as a family medicine resident. It was fast and fairly benign on the infuriation scale. Of course, the VA is apparently working with Accenture to update VistA, and are eventually looking to replace it with a commercial system. I have a feeling many VA docs will offer this to be prized from their cold, dead hands.

      And for all the griping about MUMPS, whose syntax (especially in legacy code) I agree looks like a cat walking across the keyboard, in real life on our MUMPS-based EMR it is faster and far more reliable than the Oracle-based system we upgraded from.

  28. Staggering Amount of Money by awol · · Score: 1

    As an outsider, who writes software for a living (proper, highly available transactional systems [finance industry but I do know some general stuff]). This amount of money is simply staggering. Even if we assume the published number (4.3B), 3% inflation, a relatively aggressive annualised ROI and 10 years over which to apply the costs, that turns into between 80 and 160 [20% ROI to 10% _annual_ ROI] million dollars per year in costs. IN COSTS. Even if you margin those costs at 33% (profit is already accounted for so the margin is on costs and risk) that's still 50 - 100 million dollars a year of costs to develop and support this system every year for 10 years. WTF kind of project are they planning? People have written software that changed the freaking world with a fraction of that amount of money.

    Now having said all that, I have a little window on the way a different government developed their budget for an IT project. They knew that the new project would make 60 people redundant so they looked at the cost of those people, multiplied it by some number of years for the scope of the new system and went... There you go 30 million dollars.

      There is something very, very, very wrong with government.

    BTW, There are about 20M veterans in the USA, give em all 200 bucks and let them keep scans of their own records on a freaking thumb drive. Backed up to, S3 or something. That might even actually work!

    --
    "The first thing to do when you find yourself in a hole is stop digging."
  29. Cheaper than licences tomake OS S/W interoperate ? by fygment · · Score: 1

    Seriously, why not have paid for licences (if required) and make the OS software interoperable with commercial systems?

    With the added benefit that the result would mean the DOD and VA systems would be compatible !

    --
    "Consensus" in science is _always_ a political construct.
  30. 95% tells us nothing by jsepeta · · Score: 1

    If they were only planning to support legacy systems, of course that money is designated to support legacy systems. The only way to drop it from 95% is to increase the budget and add other line items, such as new systems.

    --
    Remember kids, if you're not paying for the service, YOU ARE THE PRODUCT THAT IS BEING SOLD.