Slashdot Mirror
NYT: Healthcare.gov Project Chaos Due Partly To Unorthodox Database Choice
First time accepted submitter conoviator writes "The NY Times has just published a piece providing more background on the healthcare.gov software project. One interesting aspect: 'Another sore point was the Medicare agency's decision to use database software, from a company called MarkLogic, that managed the data differently from systems by companies like IBM, Microsoft and Oracle. CGI officials argued that it would slow work because it was too unfamiliar. Government officials disagreed, and its configuration remains a serious problem.'" The story does not say that MarkLogic's software is bad in itself, only that the choice meant increased complexity on the project.
Who owns this company?
how much do they contribute to XXX???
There has got to be some reason that this DB that ive never even heard of (and i work with DBs, its not my main point of work but I know my way around DBs) got the gig over the more established players.
or, perhaps they went with it because it is less known and therefore reduce the risk of known attacks in other DB systems?
have you seen my sig? there are many others like it but none that are the same
FTA: "An initial assessment identified more than 600 hardware and software defects — 'the longest list anybody had ever seen,' one person involved with the project said. "
Strikes me as none of these people seemed to have ever worked on large projects before.
You could take a handful of proven DB technologies such as Oracle/DB/MSSQL, throw a web (Apache/IIS) and app (.Net/WAS/Jboss) front end to it, and it would work. Why did these guys fuck up the whole thing? It's like the scene in The Fountainhead when the second-rate architects smash up the plans and add their own stuff, "to express their own individuality". This could have been a solved problem - hell, it WAS a solved problem.
I want to delete my account but Slashdot doesn't allow it.
A little googling turns up that MarkLogic's offering is NoSQL. Without getting into the whole SQL/NoSQL debate, I can't help but noting that this is clearly relational data with a fairly limited number of records (clearly there can't be more than 300M people listed) and for which ACID is (or should be) a major concern.
And here is a fantastic example of what happens when hype trumps common sense. NoSQL is the new hawtness, and apparently the dumbasses running the project wanted to be part of that. Now MarkLogic, and NoSQL in general, will have a massive blow to their reputations, and it's unknown how badly this will hurt them.
As someone who has done databases for a long time, I have very little respect for NoSQL, but that is mostly because everyone keeps trumpeting it as a killer of traditional databases. There are scenarios where NoSQL systems are an ideal fit. However, NONE of those scenarios require data to be very reliably stored in a guaranteed and predictable way.
If you don't get your tweets or your friends facebook posts as soon as they are posted, no one will really care. But for something as truly important as health insurance coverage? Are you f__king kidding me? And that's just from a reliability standpoint. Nevermind the fact that NoSQL is currently at the wild west stage where nobody is compatible with anybody else, there is nothing resembling a standard set of APIs between products, making it very difficult to develop expertise.
Sounds like the Gov was just begging for problems.
"Some people, when confronted with a problem, think 'I know, I'll use XML.' Now they have two problems."
-JWZ
MarkLogic is an XML database, not a relational database, so if your data primarily consists of XML content then it's the right tool for the job. Sounds like the vendor building the system had a favorite hammer and decided that a rather traditional database problem looked like a nail.
MarkLogic itself is fine if your data fits neatly into an XML schema, but with healthcare.gov that tree is probably enormous and hard to optimize for DB activity.
Software Shouldn't Suck
E-mail: frank at jacquette dot spamless com (remove the spamless!)
Bullshit. Since the law has been passed almost the entire implementation has been up to the administration. Which in case you had not noticed is lead by partisans in favor of the affordable care act.
If the GOP can be accused of anything material in terms of interfering with the implementation it would preventing the law from being amended, and if it needs amending there is your proof it was bad law to begin with. Sorry Obama owns this and really HE owns it not even the larger DNC as its HIS branch of government that has been responsible for implementation from the moment HE signed the thing.
*IF* it fails its either because it was unworkable in the first place as written and should not have been enacted or because the Obama administration failed to execute. Those are really the only honest high level conclusions.
The argument well because they had to pass the Senate bill as is it did not get the usual fixes and tweaking is why its got so many problems is also bogus. They had to pass the Senate bill by abusing the budget reconciliation process to deny the minority party in the House at the time its rights; they knew if usual procedures were followed it would never have become law. So again it either should not have been enacted or the administration has failed to execute.
Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
Because Postgres exists.
The "47%" that people keep talking about are almost entirely retirees, adults going to full time college, or the disabled. The largest portion of the group are the retired people. Are you saying that people who have worked their whole like and are now retired, should not have a say in taxes?
Also the working poor - 47% pay no _income_ tax, but more than half of those people do pay payroll taxes, and even many those who don't (after deductions) still work.
They also pay sales tax, property tax, etc etc.
I worked as a contractor developing a system at FDA. It lasted for 5 years. Inside the Beltway, it's pretty much the same all over. Dysfunctional communications and ridiculous breakdown of authority not corresponding to lines of management. No accountability. Project management requirements that have never been followed by any project. No commitment to the output of requirements gathering. No requirements change control. No performance engineering. Inadequate testing. No acceptance process by the government. IT groups with oversight for contractor output that have never written a line of code. All in all, pretty sick and ugly. Prior to my project there were 5 failed attempts. My project followed PMI practices, worked them hard and succeeded.
Sorry to break it to you but that "statistic" is a bunch of BS. Everyone in the US pays taxes of some sort. I'm not particularly high on the tax scale (about $36k a year) and I estimate that i pay at least 30% of my income in local, state, federal and specific taxes (sales, gas, property, vehicle, etc). MAYBE some very poor (or very rich) people can avoid some of those taxes but EVERYONE pays at least a few of them.
It is NEVER the tool, but who and why it was chosen.
Furthermore, MarkLogic is a legitimate NoSQL vendor with strong ACID.
So the question is "how does MarLogic screw up the site", without the answer to that question, we should all refrain from pointing finger to merely a small piece in a huge software project.
NoSQL is NOT the new hotness, it's been out there for at least 5 years and many successful projects are using them, so for the ones that things NOSQL shouldn't be used, wake up and breath some fresh air.
But how would a traditional relational database scale to the 1 billion, or 1,000 billion users, huh? Did you think about the need to future-proof the application?
Current population of the U.S. is a little over 300 million. That includes children, people who have company provided health insurance, etc. who don't need to access HealthCare.gov so that the number of users of HealthCare.gov is expected to be about 30 to 45 million people.
The only way HealthCare.gov would need to support 1 billion or more users would be if we inflicted it on the China or India. That could lead to war or poor technical support and customer service. Although the later does raise an intereting question of who does someone at a call center call when they need technical support? Likewise, the population of the planet is between 7 and 8 billion. For the 1,000 billion users are you planning on also providing health insurance to E.T?
Cheers,
Dave
They that can give up essential liberty to obtain a little temporary safety deserve neither safety nor liberty.
Ben
I've used it, personally, to implement a public-facing website. That site endured the dreaded 'slashdot effect' several times. No failures.
When implemented properly, Marklogic is damn near unkillable. It will slow down, it will reject connections when queues are full, but it will not fall over. Naturally, this assumes proper underpinnings and capacity calculations. With Marklogic, those are actually documented.
Mandatory disclosure: I do not have and have never had any association with Marklogic other than a paying customer.