Slashdot Mirror


NYT: Healthcare.gov Project Chaos Due Partly To Unorthodox Database Choice

First time accepted submitter conoviator writes "The NY Times has just published a piece providing more background on the healthcare.gov software project. One interesting aspect: 'Another sore point was the Medicare agency's decision to use database software, from a company called MarkLogic, that managed the data differently from systems by companies like IBM, Microsoft and Oracle. CGI officials argued that it would slow work because it was too unfamiliar. Government officials disagreed, and its configuration remains a serious problem.'" The story does not say that MarkLogic's software is bad in itself, only that the choice meant increased complexity on the project.

13 of 334 comments (clear)

  1. follow the money by ganjadude · · Score: 5, Insightful

    Who owns this company?

    how much do they contribute to XXX???

    There has got to be some reason that this DB that ive never even heard of (and i work with DBs, its not my main point of work but I know my way around DBs) got the gig over the more established players.

    or, perhaps they went with it because it is less known and therefore reduce the risk of known attacks in other DB systems?

    --
    have you seen my sig? there are many others like it but none that are the same
    1. Re:follow the money by Anonymous Coward · · Score: 5, Insightful

      from wikipedia,

      The company’s main product, MarkLogic Server, is an XML based database management server.

      This sentence has basically summed up the entire problem.

    2. Re:follow the money by Desler · · Score: 5, Funny

      They should have just used MongoDB. I hear it's web scale. *ducks*

    3. Re: follow the money by MyDixieWrecked · · Score: 5, Insightful

      I used marklogic when I worked at a previous job and after learning how it worked and understanding it better, it made our jobs incredibly easy. It just had a serious learning curve.

      Marklogic is a nosql db, that uses XML for its object format and xquery for its query language. This thing is NOT mongodb. It actually works really well and allows for complex data modeling with the ability to do joins and have transactional isolation in making changes to the data as well as a really solid content processing framework with pipelining and all that jazz.

      Now, I can't imagine a reason for using marklogic, or any non-relational db for a project like this. The only clue is that marklogic has a lot of government contracts; mostly for the military. So maybe that's why it was used. But the fact that they chose a database system that they weren't experts in for a project that had so much visibility speaks volumes on how mismanaged this whole project was.

      --



      ...spike
      Ewwwwww, coconut...
    4. Re:follow the money by mlts · · Score: 5, Insightful

      For them, it is a wise choice. It brings job security using a non mainstream RDBMS, which means that the data would take man-years to be migrated to Oracle, DB/2, or even MS SQL server which may not have the horsepower of a SPARC or POWER7 box, but one can always partition and cluster the RDBMS. It also is expensive to keep maintained, so it provides additional job security, as the expertise for this architecture is rare.

      Great win for the contracting company. Big loss for everyone else.

    5. Re:follow the money by PapayaSF · · Score: 5, Insightful

      It definitely explains the problems they've been having.

      But this is only part of it. Check out this diagram. I'm no expert, but look at all the government systems in the upper left that Healthcare.gov is supposed to communicate with, in real time. And on the right, 50 state Medicaid systems. And at the bottom, all the insurance companies. Far, far simpler IT projects have failed. This site will not be ready by the end of the month as promised, and there is a good chance that it will never work as planned.

      --
      Q: What does the "B." in Benoit B. Mandelbrot stand for? A: Benoit B. Mandelbrot
  2. Noobs? by Anonymous Coward · · Score: 5, Insightful

    FTA: "An initial assessment identified more than 600 hardware and software defects — 'the longest list anybody had ever seen,' one person involved with the project said. "

    Strikes me as none of these people seemed to have ever worked on large projects before.

  3. MarkLogic is an XML repository, not a RDBMS by CyberLeader · · Score: 5, Interesting

    "Some people, when confronted with a problem, think 'I know, I'll use XML.' Now they have two problems."

    -JWZ

    MarkLogic is an XML database, not a relational database, so if your data primarily consists of XML content then it's the right tool for the job. Sounds like the vendor building the system had a favorite hammer and decided that a rather traditional database problem looked like a nail.

    MarkLogic itself is fine if your data fits neatly into an XML schema, but with healthcare.gov that tree is probably enormous and hard to optimize for DB activity.

    --

    Software Shouldn't Suck

    E-mail: frank at jacquette dot spamless com (remove the spamless!)

  4. Re:MarkLogic = NoSQL by Anonymous Coward · · Score: 5, Interesting

    A customer at work has a MarkLogic database. I don't know what version it is, but I assure you IT IS HORRIBLE. It's like... an XML database or some shit. Just awful, and extremely confusing to use.

    A couple years ago I was asked to do automatic database backups. The only documentation I could find for backing up the database requires logging into a web interface I had no idea existed on an obscure port and clicking a backup button. I literally had to write a perl script to fake a browser and do this just so they could get a database dump.

    Do not use this product. Please.

  5. Re:Stop with the excuses. by DarkOx · · Score: 5, Insightful

    Bullshit. Since the law has been passed almost the entire implementation has been up to the administration. Which in case you had not noticed is lead by partisans in favor of the affordable care act.

    If the GOP can be accused of anything material in terms of interfering with the implementation it would preventing the law from being amended, and if it needs amending there is your proof it was bad law to begin with. Sorry Obama owns this and really HE owns it not even the larger DNC as its HIS branch of government that has been responsible for implementation from the moment HE signed the thing.

    *IF* it fails its either because it was unworkable in the first place as written and should not have been enacted or because the Obama administration failed to execute. Those are really the only honest high level conclusions.

    The argument well because they had to pass the Senate bill as is it did not get the usual fixes and tweaking is why its got so many problems is also bogus. They had to pass the Senate bill by abusing the budget reconciliation process to deny the minority party in the House at the time its rights; they knew if usual procedures were followed it would never have become law. So again it either should not have been enacted or the administration has failed to execute.

    --
    Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
  6. Re:NIH syndrome by Okian+Warrior · · Score: 5, Insightful

    Why did these guys fuck up the whole thing?

    All actions by Federal government players seem rational when viewed from their point of view.

    The problem with most analysis, here or on other sites across the web, is in the assumptions. Specifically, if you assume that the government actions are for some benefit to the people, or efficiently completes some task (which is assumed to benefit the people), then nothing they do makes sense. That assumption makes for good outrage which can attract readers, but it doesn't answer the question of "why".

    When government actions are viewed in light of more obvious motivations, everything it does makes sense. You start to see them not as an pack of incompetent, bureaucratic screw-ups, but as a self-sustaining, self-absorbed, engine of corruption.

    I'm not aware of *anything* the federal government's done in the past two decades that's been in the interests of the people. All the useful stuff, the stuff you would want to keep across a reboot, was several decades ago.

    I don't know the specifics of why an obscure database was chosen over the obvious players, but the reason wasn't "best utility for the people". Look for another reason.

  7. Re:NIH syndrome by jythie · · Score: 5, Insightful

    Sad thing is, much of the behavior one sees out of federal contracts is due to taxpayer groups demanding anti-corruption measures. A great deal of the bureaucracy comes directly from people complaining about waste and demanding a complex audible process.

    I would actually put forward that the problem is not the government, our government is pathetically weak in many ways. The problem are all the private groups flexing their muscles and making the government dance. Most of the people I have worked with on these kinds of projects are good people who just want to get stuff done, but they are weighted down by a nearly impossible set of requirements, many of which are mutually exclusive, dropped on them by outside groups who mainly are interested in showing off their power to their own people and have no investment in the project itself being successful.... and often have an active interest in the project failing since they can always use that as fuel for more personal power.

  8. US Federal Government Systems Projects by vrhino · · Score: 5, Interesting

    I worked as a contractor developing a system at FDA. It lasted for 5 years. Inside the Beltway, it's pretty much the same all over. Dysfunctional communications and ridiculous breakdown of authority not corresponding to lines of management. No accountability. Project management requirements that have never been followed by any project. No commitment to the output of requirements gathering. No requirements change control. No performance engineering. Inadequate testing. No acceptance process by the government. IT groups with oversight for contractor output that have never written a line of code. All in all, pretty sick and ugly. Prior to my project there were 5 failed attempts. My project followed PMI practices, worked them hard and succeeded.