HealthCare.gov: What Went Wrong?
New submitter codeusirae writes "An initial round of criticism focused on how many files the browser was being forced to download just to access the site, per an article at Reuters. A thread at Reddit appeared and was filled with analyses of the code. But closer looks by others have teased out deeper, more systematic issues."
This article is dated oct 8. I had assumed it would be more recent.
Doesn't it strike anyone as odd that the Govt can design and implement a billion+ dollar data storage center for the NSA but can't deploy a website to allow people to sign up for insurance?
It was slow to load, I couldn't sign up, my browser hung waiting on lost connections with the too many other files it was trying to download and there seem to be server sync problems with the back end databases.
In other words it acts like PayPal, Google, Facebook and Slashdot.
Need Mercedes parts ?
Not "systematic."
Regardless of "what went wrong", you know that the higher ups will just fire some peons, give themselves some big bonuses, and call it a day.
But the BIGGER question I don't see anybody asking, is why is there no apparent fall back or concession to delay requirements due to the problems? ANY significantly complicated computer system can reasonably be expected to encounter problems at deployment. And despite what the talking, drooling, blathering heads on TV seem to think, it is simply IMPOSSIBLE to test a system like this 100.000000000000% against real world scenarios. There will be glitches, there will be people who can't use the systems, there will be all sorts of "people problems" that no technology can fix. They should have been ready with other non webby ways to get people taken care of, and prepared to delay the needs for all of this if they could not get everyone taken care of in time.
It's hard enough to work with one spotty vendor, let alone 55. That number, 55, represents somewhere between 55 and 55-squared lines of possibly iffy communication between possibly iffy organizations. When I first heard that healthcare.gov had 55 contractors working on it, I was surprised that the damn thing ran at all.
By:
You can hold down the "B" button for continuous firing.
All they do is sabotage everyone else's work
Pretty much. It's the "starve the beast" philosophy and strategy. Sabotage something, then point out how it doesn't work, and then say "well, duh, because all government is evil."
It's their raison d'etre and since the Republicans are so invested in it after 30 (40?) years, without it they would have an existential crisis that would end in the same fate as the Whigs.
--
BMO
"free market principles" won't help here. On the contrary, just think of the money that would go into actual health care if the government came in guns-ablaze and forcefully said "no, United Health Care, you can't treat your customers like the deepest turd of a batch of untreated sewer sludge", or "no, big drugmaker, you can't throw millions of dollars on advertising niche products like fucking Restasis all over primetime tv instead of putting the money toward cutting the costs of life-saving meds".
Those are two cases where I'd actually be elated to see the NSA and TSA put into use: snoop on the moneyed fuckers involved and No-Fly 'em as soon as it's clear they want to take anything that resembles a business trip to plan their next splurge.
You can hold down the "B" button for continuous firing.
The ObamaCare web site is an example of Splat Programming. What is Splat Programming? Cut and paste from every where, run once and move on if it appears to even marginally work, and don't think very long about method or variable names. The most important part about Splat Programming is that you don't try to combine css or js files but rather just reference them individually via CDN and only change function name or variables that conflict. Most importantly, do not do any loading, scaling or security testing especially if you know that the test will fail.
The other part is Government Projects. You don't have to worry about errors and omissions because the standard government contracts do not hold the contractor liable if the final result is approved. Finally, unlike commercial projects, there is an infinite amount of money available to pay for years of bug fixes and upgrades.
Thankfully this site only effects a small percentage of people so there is really no cause for alarm.:)
Typical of what happens when an organization is too used to spending other people's money. It's ike a 16yo girl's runaway spending habits with daddy's credit card...and she's got him by the balls, too, along with her mother.
First and worst, politicians were involved. Everything else pretty much is a cascade effect off that.
Second, cronyism.
Third, you had a bunch of non-technical people setting up moving goalposts for the technical people to hit, with regard to the technical specs of the site.
Fourth, distinct lack of firm, single-message communication to the technical teams with regards to whether the project was or was not going forward.
I could go on and on about all the fuckups with regard to this. But I'd just piss off a bunch of people who aren't worth my time.
Chas - The one, the only.
THANK GOD!!!
Maybe it's the fault of libertarians that seem to make up a significant percentage of the tech demographic; wanting to kill the Affordable Healthcare Act. Or tea party programmers wanting the same thing who managed to get on the project. Come on man! Think of some more conspiracies!! Lovin' it.
Of course it couldn't be the incompetence of contracting companies that seem to make a living because they have or aim to have some sort of inside track in Washington rather than the chops to do the actual thing that needs doing. Of course that would never happen in Washington or any other political capital. I'm not saying the way the primary contractor, Quebec company CGI, does business in any way follows recent Quebec business practices. They are probably a well above board and good honest corporate citizen (although according to the Washington Post article above they did screw up another medical system based project). I'm just saying that if Quebec ever did separate from Canada, as it is now, they'd have to think up some other adjective to describe it. It's too cold to grow bananas there.
Frankly (and personally) though, I wouldn't trust any company to government contracts with stated aims published in their profiles like: "The ultimate aim is to establish relations so intimate with the client that decoupling becomes almost impossible," (see Washington Post article). Especially not from Quebec.
-- I ignore anonymous replies to my comments and postings.
One could argue that the Administration's tactic of preventing release of critical design data until after the election, to prevent the opposition from using the true costs as a campaign issue, was sabotage de facto. This put the entire development process several months, perhaps a year, behind schedule.
It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
Actually, Social Security is hardly bankrupt. It has about 3.5 trillion dollars invented in special interest drawing T-Bills. Unfortunately, the deadbeats in Congress borrowed the money "invested" by Social Security and spent it on every Congressional wet-dream and war they could come up with.
The "full faith and credit" of the US requires that they pay this money back. This means raise taxes, run the printing press, or weasel their way out of as much of the repayment as they can. Every dollar they actually have to repay is a dollar that can't be spent on future corporate welfare.
When you are dancing with wolves, never limp
I will argue that part of the political problem is the boomers (of which I am one) - we grew up spoiled, filled with neo-socialist propaganda (see "The Closing of the American Mind" by Alan Bloom), and isolated without much chance to learn how to get along with each other or to how to be spouses and parents. For example, never having had to share a bedroom meant we never never really learned the art and necessity of compromise and living with someone else. We're arrogant, self-centered and always convinced we are right about everything. So, now we are running the political system, it is inherently dysfunctional. And that's not even counting those of us who are still lost in the 1960s, and think the hippie utopia was the best of all possible worlds, disregarding the realities of life. Someone once described American liberalism as confusing wishes with facts.
So, politics in the US at least will continue to be dysfunctional until we boomers age out of the power structure. Assuming the next generations aren't even worse... :P
It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
Quote 1: "A complex system that works is found to have invariably evolved from a simple system that worked. . . .A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system." (John Gall, Systemantics,p. 80, 1978 paperback edition).
Quote 2: "In architecting a new [software] program all the serious mistakes are made in the first day." (Martin, 1988, cited in Maier & Rechtin, The Art of Systems Architecting (3rd ed.), p. 399)
Quote 3: "Indeed, when asked why so many IT projects go wrong in spite of all we know, one could simply cite the seven deadly sins: avarice, sloth, envy, gluttony, wrath, lust, and pride. It is as good an answer as any and more accurate than most." (me, testifying before the Subcommittee on Government Management, Information, and Technology Hearing, US House of Representatives, June 22, 1998)
My pre- and post-launch analysis of the Healthcare.gov website can be found here. ..bruce..
Bruce F. Webster (brucefwebster.com)
What went wrong? Government.
The ACA has some great theory behind it. Assuming that the federal government will be able to operate and maintain a system like this in a cost effective fashion is lunacy. It as bound to fail.
Also don't tell me it was Republican "starve the beast" strategy. The ACA was fully funded and largely untouchable. By any reasonable standard the roughly $400m spent on implementing this was incredibly excessive. If a private company had wanted to build this system for profit, it would have been done for under $100m. The big mistake of the ACA was that it did not allow for the creation of privately run and owned exchanges.
From Kevin Drum's blog:
So basically, these insurance companies sending out these cancellation notices were gaming the system so that they could both undermine the law and blame it for "forcing" their customers to buy more expensive coverage.
Reagan? He started the downward spiral toward total dishonesty and lack of government. Why are some countries rich and others poor? It's not resources, it's government. Germany and Japan had good systems imposed by the US and they are doing fine. Most Western European governments have similar systems, they do well. Most Asian/African/Eastern European have crappy governments and are poor. There are exceptions, Singapore has a repressive but effective government and they are doing just fine economically.
By choosing to starve and neuter the most effective tool for prosperity they have Americans are making themselves, and the countries who follow them poor. Government, and taxes, are a good thing. Corruption is bad, but a little theft is better than selling out the whole system which is what the US has consistently done for the last 3 or 4 decades. Who won each election? The man was bought. Why did Clinton win? He sold out more completely than his opponents. Why did Bush II win? He sold out totally and without reservation. The one exception is Bush I who actually did some positive stuff before being run out of town on a rail for not being bought. Obama was sort of a mistake, it should have been Hilary who was utterly bought, but Obama did the grassroots thing the first election... Too bad he doesn't understand Texan aphorisms like "dance with the one who brung you."
Government is good, Fox news sucks, current conservatism (here in Canada too, Harper is trashing the economy in the typical right wing manner) sucks, propaganda sucks, and going with the gut instead of what works (the economy was better when taxes were high? That can't be right...) sucks.
You got me into this! You were the ideologue! I'm only a poor assassin! - Twenty evocations, Bruce Sterling
Why does everyone think making a web site is easy? With multiple feeds using different technologies even a fairly minimal health care web site would be complicated. Add in a whole lot of states that oppose the process and delay finalizing the requirements (client from hell) and you can pretty easily get to a point where the implementers have to choose between being late and being wrong. Think of the length of the requirements document distilled from the laws and negotiations. Think of the army of business analysts needed to get functional requirements and of the timeline they have to meet. Remember that no one ever hires enough business analysts.
This is not an easy thing to do.
You got me into this! You were the ideologue! I'm only a poor assassin! - Twenty evocations, Bruce Sterling
Use your special system architecture x-ray vision, folks. This is not simple, stand-alone site like Slashdot that just has to do some database queries and generate some XML, then uses JQuery or something to asynchronously load some advertising into a DIV. This is a system that must orchestrate a complex *synchronous* process involving servers that belong to outside organizations.
Case in point; the system requirements say that the site must exclude illegal immigrants, so the system has to request and obtain proof of your status from Homeland Security's servers before it can proceed. Also, instead of issuing the same subsidy to everyone, the law specifies and income dependent, means-tested subsidy, which means the system ALSO has to check your claims against the IRS's computers before continuing. That's before it actually gets to obtaining the marketplace data.
So the most complex aspect of this system is essentially untestable short of a near-full scale roll-out. Hey, IRS, can I try hosing down your servers with JMeter? Even if you could orchestrate the non-functional testing you'd want to do, you won't know how the system works until it's handling real data. It's not like you can shove a test load equivalent to a thousand applications per hour, then another equivalent to ten-thousand, then draw a straight line that will tell you how the system will perform with twenty-thousand. There are some serious discontinuities in performance lurking, and the actual data submitted is likely to change things.
I think if I were in charge of this, the extreme difficulty of realistic non-functional testing might have led me to isolate some of the data interchange into a post-processing step. That is, I'd let people apply and take them at their word about their immigration status and income, then tell them to check back in a day while we confirm the data they submitted. It's more bureaucratic, but a big part of user experience is predictability. If someone knows they can complete their application in half an hour and come back 24 hours later for confirmation, it's not so bad. But if the system is designed to give them the expectation that they can finish in a half hour, but sometimes takes so long their sessions expire, that's a disaster.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
Read for yourself the actual regulations, published in mid-2010, for grandfathering of existing plans. Less than 35 pages of single-spaced small print, so not too hard of a slog as these things go. A few recommended highlights:
In short, it seems clear from HHS's own pen that the concept of "grandfathered" plans under the ACA is (1) highly Orwellian; and (2) was deliberately set up for failure. It's disappointing that the latest distracting meme is blaming the insurance companies for doing what, as shown above in black and white, HHS fully intended to force them to do from the beginning.
Grant me the legal authority to print money anytime I want and make everyone else pay the true cost of it (inflation) and I, too, could pay for anything money can buy. In the Apollo days they at least tried to pretend that debt is important and that there's something deeply wrong with running a government in a way that would bankrupt any business or household.
Hold your horses, partner.
A history lesson is in order. (Then get off my lawn.)
The 1960s had a lot of debt.
There was the Vietnam war and it wasn't cheap. There were some questionable political deals in Cuba that included a rather scary nuclear showdown that led directly into the cold war. Also there was the whole space race that you mentioned.
The US was in debt and facing a deficit. Not as big as today's deficit and debt, but it felt bad at the time.
President Johnson was looking over where the money was sitting, and he noticed a huge pile of cash sitting in an off-budget area. It was called the Social Security Trust Fund. It had billions of dollars just sitting there being invested, not being spent.
The good president looked over the budget, noticed that he could make himself look better (and presumably look better on the world stage) if the US didn't appear to be in debt. So President Johnson decided to move the Social Security Trust Fund into the general budget. There was a bit of a complaint at the time, "you cannot spend that money, it is for retirement". Not a problem they assured us, there would be plenty of money available in 2010 when baby boomers start to retire. We might not even be on a cash society in the future, let's spend it all today! The President made a proposal to Congress, and then all of them started rolling up the Social Security funds into cigars and enjoyed a smoke.
The Apollo program and several other major programs were funded by TODAY'S social security problem. Much of the reason we have so much debt is because the social security fund was robbed to pay for the war and the space race. Government took out a loan from the people and only recently started feeling the pain of paying the loan back. Baby boomers who don't suffer from society's generally short term memory can clearly recall that the focus was divided on the war, the protests, and the space race, and how those few people who noticed the money was missing were quickly written off as being anti-war or pro-war (whichever was a better distraction) and somehow the messenger was blamed and the message quickly forgotten.
Much like groups like WikiLeaks today; we all remember the name but the hundreds of soldiers who were documented committing clear acts of murder somehow escaped the court martial. Back then if you mentioned the social security funds you were branded a hippie or communist and you didn't believe in America. (Anything to make you look like an unpatriotic troublemaker rather than someone who wanted to see where the money went.) Then Johnson lost to Nixon and another scandal followed, most people forgot about Johnson's scandal taking the money and moved on to Nixon's spying scandal that evicted him from office, which is NOTHING compared to today's spying scandal that people don't care about.
Enough rambling, get off my lawn.
//TODO: Think of witty sig statement
Correct. It was a no-bid contract. Interestingly, Toni Townes-Whitley, a senior vice president at CGI Federal, is a Princeton classmate of Michelle Obama. In addition to being college classmates, both Obama and Townes-Whitley are members of the Association of Black Princeton Alumni.