Slashdot Mirror
Cost of Healthcare.gov: $634 Million — So Far
First time accepted submitter Saethan writes "Healthcare.gov, the site to be used by people in 36 states to get insurance as part of the Affordable Care Act, has apparently cost the U.S. Government $634 million. Not only is this more than Facebook spent during its first 6 years in operation, it is also over $500 million above what the original estimate was: $93.7 million. Why, in a country with some of the best web development companies in the world, has this website, which is poor quality at best, cost so much?" That $634 million figure comes from this U.S. government budget-tracking system. Given that this system is national rather than for a single city, maybe everyone should just be grateful the contract didn't go to TechnoDyne.
Money != contractor knows what it's doing
Should have mentioned that they make their money off of inventing problems after the contract is signed.
Technoli
The site had how many people try to sign up in the first day? If you want to compare it to facebook (a popular metric here no doubt) the number of people who attempted to access and sign up on healthcare.gov in the first day dwarfs the first several years of enrollment at facebook. If they had attempted to build a website to handle the load they faced (which will of course taper off quickly once the first wave of enrollees are signed up and done shopping) we would be bitching that they overbuilt the site because they would have tons of servers sitting mostly idle after the initial surge is done.
We need to wait until it has been up for a while before we go around calling it a failure.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
In other words, the issue right now is not the cost of the thing but whether any amount of money can make it healthy in the required time.
If this thing doesn't get right, "they" might have to wave the fine/penalty/tax to be payed by people who didn't sign up, which is why there is a political fight right now "shutting down the government"?
This figure is not just for building a website.
It is for all spending with CGI Federal over the time that they have been doing business with the Federal government, including payments from fiscal years before Obamacare was even passed.
The figure is now being regurgitated by various right wing websites without anything that even passes for thinking.
And also now slashdot, which is disappointing.
It's called 'The Cloud', you can buy additional instances that first month and get rid of them when you no longer need them.
That figure covers 114 separate contracts (see http://usaspending.gov/explore?tab=By+Prime+Awardee&fiscal_year=all&idvpiid=HHSM500200700015I&typeofview=transactions ) Not to suggest that it still wasn't overly expensive, but consider the fact that the system is a national transaction application that has to dip into numerous other federal data sources - and has a mission criticality above and beyond facebook. Still, many of us could have done it better and cheaper, but then again very few of us would actually enjoy working for the federal government and conducting our business the way any federal contractor is required to.
-- Religion is not an exact science
The solicitation number linked to actually refers to the HITECH act, part of the American Recovery and Reinvestment Act, to quote health it.gov:
The Health Information Technology for Economic and Clinical Health (HITECH) Act seeks to improve American health care delivery and patient care through an unprecedented investment in Health IT (HIT).
And it certainly sound like they've achieved an unprecedented investment at least.
Facebook as a privately held corporation for its first six years can cherrypick the cost of its infrastructure as it sees fit. cheap and powerful infrastructure is always a very warm prospect for a market that may be keen to see returns from a soon-to-be public company.
Facebook doesnt take into account the fact that its final cost is spread across the backs of millions of FLOSS developers its never known, whereas the US government is literally developing a system, an open market, that has never existed outside of a single state in its union. The government also doesnt attract facebook-level talent and as such is forced to contend with best practices as it outsources development to well-established industry players. the government began much larger and more fiscally sound than Facebook in its first year, so the purse strings are of course looser.
you're comparing a private company with independent autonomy in the software lifecycle to a government agency beset with lobbyists and average, but not astounding talent. in some cases edicts instituted by governing bodies of the program which may mandate outsourcing to specific vendors regardless of cost; this is how politics works in both private and public sectors. im also certain the signup rate for facebook in its first six years is dwarfed by the healthcare site in its first six hours, which may help explain some of the cost of the program overall. keep in mind the estimate of ~90 million may have been an intentional underestimate as the reform had to be sold to a congress that would rather see the president dead than re-elected.
Good people go to bed earlier.
They could have hired some friends of mine, who are expert web database interface programmers, plus me (I'm fantastic at proper HTML and CSS styling and future-proof coding) for about $500,000. Where $90+ mil came from is beyond me and the $600+ mil should put some people in jail for fraud. Heads better freaking roll over this atrocity, which is probably now covered by health insurance.
When a site loads 50+ .js files after you click an 'Apply' button, something is wrong with the design.
It seems to me that the larger the bill and the larger the company sending that bill, the lower the competency.
Our three-person company handles web sites serving hundreds of thousands of users per day for a few thousand dollars. We could easily handle a few million users by adding a few more database servers at a cost of around ten thousand.
Handling what is potentially HIPAA-covered data? Much harder to do than just working with credit card information.
Almost 7 times over budget. And it didn't handle the load placed on it. 8 days later and it's still having problems. And you want to defend it? Oh it's ok that it's a huge steaming pile of crap because why exactly? Do you work at CGI Federal? I could see if it came in on budget. But even then, they obviously did not do any research into how many people would be interested in the site.
"Cowardice in a race, as in an individual, is the unpardonable sin." --Teddy Roosevelt
One website, that's expected to have incredibly heavy loads, will handle personal medical and financial information, and must play suitably well with a ton of third-parties' services while being the target of severe attacks from any foreign government or script kiddy who doesn't like it..
You do not have a moral or legal right to do absolutely anything you want.
I would like to know if there are any connections between contractors and those awarding the contracts - ie Family ties, business connections, etc... in this day and age there is absolutely no way a website should cost this much. I team of around 20 proficient Web professionals should be able to make almost anything in around 1-2 years max, with a max cost no more than 10 million. Half a billion? Follow the money, this is at best gross negligence on the part of those awarding the contracts, at worst misappropriation.
Active duty military who get paid very little to defend the country, and VA staff.
Not so, It is not "padding" per se. However, this is the general way federal government acquisitions work (at least in the DoD). Staff member gets 3 quotes from vendors and submits to contracting office. Contracting office goes to their GSA-approved buddy. GSA buddy sends purchase request through 3 layers of GSA approved subcontractors. Each layer adds their markup. Last one in the chain ships product to staff member at highly inflated price. Each layer of GSA-approved vendors get their cut for doing nothing (except the last guy who shipped it), and the product cost 3 times as much. Now, contracting officers have nice new job waiting for them upon retirement from civil service, and free cash was distributed to those who can game the GSA system.
Except that there are services nowadays SPECIFICALLY built for this type of scaling, like AWS. You can spin up extra servers for temporary high traffic - especially high traffic that was absolutely foreseen. Funny how Amazon's website can handle the traffic on black Friday just fine.
Sorry, but I've been doing web development for 15 years and have worked on large projects. I can't see the cost for this project being more than $20-30M for up-front development (that includes planning, documentation, meetings, coding and testing - all without outsourcing) and at most $20-30M per year for software licensing, hosting, bandwidth and maintenance. And I'm talking top-dollar.
For $634M, they could have gone down the wrong path (the one you mention) and committed to long term-contracts for those unnecessary idle servers and still have $300M leftover.
People who say "money does not buy happiness" are just people without money trying to make themselves feel better.
I have insurance through my employer and they have no intentions of cancelling. That said, I was curious about plans and premiums and chose to check it out. It has now been a week, and I have only been able to get to the Contact Infomation screen before i get the Unknown Error message. It took me 3 days to create an account alone without error. I have tried IE, Chrome, and finally caved into trying Firefox as well...and the site problems are not browser-related. They are coding-related. As of today, I am still unable to see actual plans or premiums...and my time is too precious to spend on the phone when there is a website that is supposed to work. I can also say, load is not the problem either. The first day or two had massive wait times on the site, now, I get in fairly quickly...only to see Unknown Error.
One: Schedule Fail. Compounded by late award of the contracts to develop/influence:
Contracts Awarded Dec 2011
Two: massive requirements base to develop specification for development and implementation: The PPACA was 1800+ pages, and the associated regulations are 10,000+ pages, and are STILL changing. Can't develop without a spec and design, with big parts of requirements still changing.
Three: inadequate testing. The above-referenced link states that security testing BEGAN in August 2013, less than two months before rollout. There's no mention of load testing
Four: Integration issues. The Obamacare Exchange system combines data from numerous agencies and systems, and integrating between them is always a difficult task
Five: Identity-management. This is in parallel to Integration, somehow all identities need to be federated into a single overarching system.
Twenty-three months, even with a top-flight team, would simply not be enough to do this: this is a 5-7 year job. . .
That's true, and a very good point. I don't work with HIPAA-covered data, but could they use something like amazon's government cloud?
One website, that's expected to have incredibly heavy loads[...]
Well here's the rub. In regular operation, the loads aren't going to be incredibly high. They'll be "very" high, but not ridiculously. You could argue that their single largest mistake was trying to do a massive roll out to everyone in the country all at once. They should have rolled out to a small number of people, worked the kinks out and come back in a month with a slightly larger roll out. Rinse and repeat until it's available for everyone and you have some idea what your actual day to day usage numbers are going to be.
Exchange launch turns into inexcusable mess: Our view
Park said the administration expected 50,000 to 60,000 simultaneous users. It got 250,000. Compare that with the similarly rocky debut seven years ago of exchanges to obtain Medicare drug coverage. The Bush administration projected 20,000 simultaneous users and built capacity for 150,000.
That's the difference between competence and incompetence.
The too-much-demand excuse also is less than the full story. In addition to grossly underestimating demand, the administration and its contractors seem to have made mistakes in building the websites. The system for verifying consumer identity has had persistent problems, as have pull-down menus.
Nor were problems confined to the 36 state health exchanges run by the federal government. Sites run by 14 states and Washington, D.C., bogged down because they have to refer to federal databases to verify consumers' identity.
much of left-wing thought is a kind of playing with fire by people who don't even know that fire is hot - George Orwell
The comparison to Facebook is complete BS.
Even though (as somebody already pointed out) the $634m number doesn't represent just Healthcare.gov, the comparison to Facebook is completely fallacious. Facebook has money coming in other than just their investments; the investment money that is referenced in the Crunchbase page is in addition to any other income that they had. In other words, Facebook spent way more than $634m in that period of time.
Lazy journalism at its best.
The real problem is that NOBODY, in ANY branch of the U.S. government, gives a shit about anything other than enriching themselves.
I cordially invite ANY evidence to the contrary.
If you are talking about politicians I'll agree with you. However if you are talking about government employees I have to tel you to taking a flying F@&K, as you have no idea what you are talking about. I am working without pay at this time. I don't know when I will be paid thanks to the shutdown, but that hasn't stopped me from doing my job.
This statement alone is scarier, than whatever was leaked by Mr. Snowden. Surprisingly, the President's cheerleaders — normally so suspect of government's invasions into our privacy — ignore this implication.
Gravity of the mission, whatever it is, has little to do with the cost of implementation. First step on the Moon was a gravely important mission, but it was easy for Neil Armstrong to do it...
Yet another argument for letting the government do as little as at all possible — rather than explode its size as the Administration is doing.
In Soviet Washington the swamp drains you.
Here's a nice overview of just what's going on with the ACA website. The chart from Xerox illustrates why the system is a just a teensy bit more complicated than Facebook. http://www.washingtonpost.com/blogs/the-switch/wp/2013/10/09/heres-everything-you-need-to-know-about-obamacares-error-plagued-web-sites/
All of which had the luxury of a slow rollout, and don't have anywhere near the same amount of damage done if they're compromised.
You do not have a moral or legal right to do absolutely anything you want.
Agreed. That would likely have worked out much better... but politically, it's impossible. Why does district X get access, but not district Y? That particular random criterion is slightly correlated with this obscure trait, so clearly the politicians in charge are working for or against those people, and don't deserve to be reelected...
You do not have a moral or legal right to do absolutely anything you want.
That's a part of it. The largest part in the evaluation is education of work force. Not a lot of rank and file programmers in the US get more than a bachelors degree. Why would they? Unless you're doing work with advanced algorithms or some sort of management there aren't a lot of drivers to have the additional education.
Because of the weight contracts have on education you see a lot of folks with unrelated degrees and foreign diploma mills. That leads to poor final output.
On a campaign level the administration knows how to put together software quickly. But that's not the way the law allows the gov't to operate. Large contractors have been gaming the bidding process for three decades.
This is pure bureaucratic inefficiency work at it's finest. Some examples if this is like a typical Federal contract would include things like:
Changing specs on what your asking for multiple times throughout. You start building to one spec and part way through things change to another spec requiring expensive redesigns. Case studies have been written and college courses taught about the sheer number of design changes on why certain federal programs that have run billions of dollars over.
Too many chiefs calling the shots which requires too many chiefs answering for the shots being called. For political purposes you can have people from any number of agencies and or divisions within an agency all trying to design the thing. Almost none of them have a clue what their doing, but they'll pretend to be a designer just because they can. The resulting quagmire can cause committee upon committee just to get things approved at any given level and in case you missed someone that feels overlooked they can bring the whole thing to a grinding halt just to remind everyone not to overlook their office.
If your the Federal Government your allowed, in fact your - required - to use racism and sexism when bidding things out. Anyone that is involved with government contracts is well aware of this and as a result contractors that meet the discrimination guidelines get selected over those that don't even when they cost significantly more. When your guaranteed to get a job even when your charging more money, do you think someone is going charge the market rate or their chosen rate?
Politics, don't forget about politics as the new administration gets in and typically wants to kills anything that was a signature of the old. If you think life is difficult with inter office politics, imagine having powerful senators and governors doing everything they can to run interference on your project.
This is only a small smidgen of reasons why these things run costs that are sky high as they are and part of the reasons why you see Republicans want to cut government spending. They look at something like this and say, the private sector would do this in a fourth the time for a fourth the cost (not taking sides, just explaining their logic).
As opposed to the health insurance industry, which is a billion dollar a year boondoggle whose only functions are to determine who gets billed for what, and to deny benefits in order to increase "shareholder value".
Even fairly incompetent governments around the world have been shown to be able to manage a single-payer system without it becoming such a drain on the GDP.
Every military person I've known have done it for the free college money they give out. The military basically buys service with tax dollars. These kids don't generally go die for free and if they do it's because they were brainwashed into nationalistic American exceptionalism.
Working at the VA is a huge credit on anyone's resume, especially in the Neurology/Medical sciences field. It's hard to get in there but a huge bonus if you can claim that experience.
People are inherently selfish but I will admit there are a small percentage that do things because they mean well and desire nothing in return. A dying breed for sure.
Der Tod ist der einzige Weg hier raus!
The plans are too expensive for poor people. This government healthcare stuff has made healthcare much, much more expensive. I mean back in the day I had a $10 deductible with full coverage for everything and I paid $348/mo. Now I'm paying $20/mo for my employer-supplied CDHP that gives me an HSA I can add $3500 per year to pre-tax, meaning I save about 30% on everything from bandages and antihistamines to Target clinic visits and emergency surgery. On top of that, the HSA covers anything beyond a $3,500 per-anum maximum: if I get cancer or break a bone or whatnot, I pay $3,500 and they pay everything (my deductible is $1500, but it goes toward this $3,500 maximum). They do pay 100% of any wellness service I contract--a doctor's visit for a check-up, physical, vaccinations, other preventative care is covered absolutely and I pay $0.
For the privilege of managing your own risk and being immune to sudden ridiculously high health care expenses beyond $5,500 per annum, while paying for absolutely everything yourself, you need to pay... $194/mo on the open market. Seriously. HDHP for an individual is about $200/mo. The cost of a plan with Wellness benefits (the insurance company pays for you to get regular check-ups and vaccinations to avoid the eventuality of paying out when you fall severely ill) and a $3,500 per-annum out-of-pocket maximum is around $300/mo. They charge you $200/mo to have a savings account and $300/mo to get two $75 doctor's office visits and a $15 flu vaccination for free.
The risk should be substantially lower if they supply clinical services. Some modern HMOs are as low as $70/mo and they have actual coverage. How in the fuck are no-coverage HDHPs $200/mo?!
Support my political activism on Patreon.
What right wing sites, liberal???
There are no right wing sites. All of the sites, and the entire media, are left wing and biased. It is a constant attack on our principles, our freedoms, and America by the entire universe, and reality, which has an unfair liberal bias! Why do you hate freedom, liberal? Why do you hate prosperity? Can't you see that there are only a few conservatives (read, glorious defenders of freedom) left, and that the brave ones who speak out are shot? GLORY, GLORY HALLELUJAH! We shall prevail in the end!
Just wait until they actually start managing your health care.
It came from middle-middle-middlemen. We've privatized the hell out of a lot of important tasks that the federal government does in the name of making them cheaper, but I think every single person in our industry can tell you that contractors are expensive as hell, and add nothing but immediacy.
So, we pay full time people in the government to review contract bids. Those contractors are specialists in winning government contracts, and do nothing other than hire sub-contractors. Those subcontractors hire actual employees, but only a trickle of the money they make goes to paying for the work. They take a huge overhead for legal, HR, actual overhead, and profits. The parent contractor takes a similar huge cut before passing things on to subcontractors.
We've already multiplied the actual costs by 10 or more, without having even brought "overruns", "missed requirements", and real QA into the picture.
But don't worry: the ACA is going to reduce the deficit.
Obama ran on the platform that something needed to be done about the millions of people that had no healthcare.
And millions of people are under-impressed by the fact that Obama signed us all up as customers for giant health insurance companies instead of actually doing something to ensure that people get something useful out of the venture.
I guess the only surprising thing is that only a million people tried to sign up. With all of the grass-roots programs encouraging people to sign up, with all of the hype, they should have been expecting traffic of DDOS proportions.
The massive health insurance company bailout act of 2010 (aka affordable care act) does not dictate that everyone has to buy insurance this week. While it does unfortunately have a mandate in it as a massive concession to the health insurance industry that contributes in huge numbers to nearly every politician in Washington, it does at least give a few months before that mandate kicks in. Hence they did not have a good reason to expect every uninsured person to log in in the first week, and indeed that did not happen.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
The problem with your idea is that this site was NOT built by the government. It was built by private contractors in a competitive bidding process.
And you want to turn the police over to private contractors?
Lots of other things are done by private contractors for the government. For example most of the defense department procures everything it gets via competitive bidding from private contractors.
I'm looking forward to the day when the federal government runs all of healthcare with their awesome efficiency and competence.
Except for ending slavery, the Nazis, communism, & securing American independence, war has never solved anything.
It's not all their Federal work. It's all work for the Centers for Medicare and Medicaid Services, which is the group implementing and managing healthcare.gov.
Ars has a great article up going into more depth of why this happens so often here: http://arstechnica.com/information-technology/2013/10/why-us-government-it-fails-so-hard-so-often/
I see this comparison a lot, so let's dive into it.
http://www.marketwatch.com/investing/stock/fb/financials
Facebook had a COGS cost of 1.36 billion, and a R&D cost of 1.4 billion. Since they basically only operate a web site, that's all the cost of operating a web site. So to 2.76 billion dollars in a single year spent on Facebook.com.
Healthcare.gov spent 614 million over three years. At $200 million a year, that's roughly in line with Facebook's spending level back in 2009.
And Facebook has never gone down, right? It's never had a load issue, right? Yeah, didn't think so.
You're of course assuming that the purpose is the same as it's stated purpose, to work, rather than funnel money that used to go to the Good 'ol Boys to the Good New Boys.
The Bottom lines is we pay $5 - 5.5K per capita for health care or $ 1.57 - 1.727 trillion, just add the DOD budget to the Medicare/Medicaid and the difference is a measley $200B of chump change; anything that doesn't gut the DOD is just liberal feel-good smoke and mirrors. Middle-class People are reporting their quotes on the exchanges are in the $500.00 a month with a $20,000.00 anual deductable range, that's just insane.
Apocalypse Cancelled, Sorry, No Ticket Refunds
Before it was scrapped, the Canadian government had shelled out over a billion dollars to pay for the federal gun registry. It was initially budgeted to cost a few hundred million. Why the bloat? Because they didn't factor in the cost of every single department and major player having a different computer system, and wanting integration with their systems, and they didn't want their individual departments to pay for it, or have to change their own internal systems. So it all got added into the registry's budget instead.
Magic doesn't work in my presence. My power of disbelief is too strong.
How many of those were 304 not modified? And how large were they? I don't know but no one else seems to except one guy who does web design, and apparently knows little to none about servers.
All of that happens client side, making his DOS comment ridiculous. He did not say that caching was disabled, and did not give a byte count. I'm discounting the whole report until I have time to look myself.
Medicaid/Medicare and Tricare/VA, the only government administered healthcare programs in the US, have been more or less doing okay all this time. Sure, there are some flaws in the VA when it comes to mental health coverage, but we're getting better at diagnosing and tracking PTSD.
Occasionally living proof of the Ballmer peak.
This figure is not just for building a website.
It is for all spending with CGI Federal over the time that they have been doing business with the Federal government, including payments from fiscal years before Obamacare was even passed.
The figure is now being regurgitated by various right wing websites without anything that even passes for thinking.
And also now slashdot, which is disappointing.
If you chase the links to the original treasury website, half of the $634 million was paid after the passage of the 'Affordable' Care Act, so I'm especially interested in the specifics of those contracts- which are still more than triple the $93 million dollar original ACA website contract.
So perhaps it's a $300 mil website instead of $600 mil. That's not really much of an improvement, to be honest.
"No, it's all lies! The website only cost a bit over a quarter-billion dollars!"
We do have to find out more specifics, to be sure. We've certainly sent that company a massive amount of money since 2010.
Alcohol, Tobacco and Firearms should be the name of a store, not a government agency.
How many of those were 304 not modified? ...
All of that happens client side, making his DOS comment ridiculous.
First, 304 headers are generated by servers when a client requests a page that has not been modified since the last access date reported by the client. Clients don't generate those headers themselves, so, no, its not all client side.
Now if you understand how DDOS attacks work (All they do is open a LOT of connections), then you'll understand that having 50 or 75 separate links in your page, even if they ALL get cached, will still cause 50 or 75 separate connections just so your server can tell the client "304: Not Modified"
That, along with the fact that most systems are setup to limit http/https connections to 2 per client at a time and that many people have some not insignificant latency, is the reason behind embedding js directly in the html source and using sprites with CSS.
I only found 4 external JS files linked in their source, not 50, but the point is that 50 js files, even at 0 bytes each, with 25 million visitors is DDOS quality.
It's not just 'one freaking website'. There's a HUGE backend with financials, a full customer management system, integrations with dozens of other systems and data sources etc. The public facing website part of it is like 5% of the work, if that.
The summary is misleading to the point where I think it's deliberate:
Lets look at "Not only is this more than Facebook spent during its first 6 years in operation"
This is worded like it's comparing the cumulative cost of Facebook's first 6 years to the ~3 years that Healthcare.gov has been in development. But they're actually talking about the annual cost of Facebook compared to the cumulative cost of Healthcare.gov. As for Facebooks annual cost Facebook spent 449M in 2010, 1.1B in 2011, and 3.19B in 2012. FB also has the advantage of a far slower rollout, dealing with far less sensitive data, and needing far less integration with other systems so it's unclear if it's a valid comparison for things other than load.
There's another whopper in "it is also over $500 million above what the original estimate was: $93.7 million". So lets look at what the article actually said:
Take that out, and you’re left with roughly $363 million spent on technology-related costs to the healthcare exchanges – the bulk of which ($88 million) went to CGI Federal, the company awarded a $93.7 million contract to build Healthcare.gov and other technology portions of the FFEs.
So Healthcare.gov was never supposed to cost $93.7 million, only the contract to CGI to write the code was $93.7 million, the rest of the numbers had nothing to do with that.
There's certainly issues with Healthcare.gov but this story looks like a partisan plant to me.
I stole this Sig