Tech Titans Oracle, Red Hat and Google To Help Fix Healthcare.gov
wjcofkc writes "The United States Government has officially called in the calvary over the problems with Healthcare.gov. Tech titans Oracle, Red Hat and Google have been tapped to join the effort to fix the website that went live a month ago, only to quickly roll over and die. While a tech surge of engineers to fix such a complex problem is arguably not the greatest idea, if you're going to do so, you might as well bring in the big guns. The question is: can they make the end of November deadline?"
Nine women cannot make a baby in one month.
So I'll be able to log in with my google account and everything will already be filled in.
Our Gov is finally "out of patience" with Vermont's site (built by the same CGI that did such a bang up job on the Fed system: http://www.burlingtonfreepress.com/article/20131031/NEWS03/310310034/Governor-Peter-Shumlin-Web-woes-prompt-changes-to-Vermont-health-reform
I think it's cavalry.
I think they should have just listed the plans on Amazon. Almost everyone already knows how to buy stuff from them and their servers would have handled it.
bombing the hell out of it!
It's a Biblical reference -- and at this rate it would take divine intervention.
Laughter is the Spackle of the Soul.
IBM certainly made sure the Nazi's CRM system worked right.
In two months the site will be using Oracle and Ellison will charge the Feds a fortune for the license fees.
Google will start mining every piece of data it can get off the website, of course the NSA will be stealing that and stashing it in Utah.
Red Hat will push it all to RHEL which isn't necessarily a bad thing.
Harrison's Postulate - "For every action there is an equal and opposite criticism"
No Microsoft? lol :)
No, they wanted it done and not outsourced to India.
Harrison's Postulate - "For every action there is an equal and opposite criticism"
As an Anonymous Coward, I am very concerned that proper language be used only when it places me in a position of higher authority.
The word for "soldiers who fought on horseback" is cavalry.
The word for "a hill near Jerusalem where Jesus was crucified" is calvary.
Just kidding.
I guess nobody in the decision making loop heard about Oracle's big California DMV fuck-up.
-jcr
The only title of honor that a tyrant can grant is "Enemy of the State."
...and if there were, it probably wouldn't involve a violation of Brooks' Law.
Brooks Law states "adding manpower to a late software project makes it later".
Hey, Windows users, there is no such thing as "forward" slash, there is only slash and backslash.
I, for one, am glad to see government doing something right. They have fallen short of privatizing the site, but....
Will the three tech giants also teach Economics?
"The mind works quicker than you think!"
He has already succeeded at that, and if ObamaCare gets off the ground it will be like 10x over all prior presidents combined.
Sig Battery depleted. Reverting to safe mode.
In that scenario, we'd actually be worse off - the ones with principles wouldn't be working on it...
Can you be Even More Awesome?!
The choice of these companies makes it obvious that it is not an asp.net fix. Being from Canada I have no idea what front end the site is using in the first place. But if it is not a based upon Microsoft style asp.net in the first place then you can bet that the choice of who gets government contracts will be effected in the future.
Here in Canada the government has completely sold out to Microsoft and in some cases if you need to access government services on the net it is all coded in asp.net especially the revenue Canada sites where you do your taxes. I find it hard to believe that a Microsoft software based contractor did not get the original site contracts in the USA in the first place. Again if the site is not fixed on time then you can bet Redmond will have a PR field day with this one, if it is Google, Oracle, and Red Hat fixing asp.net code then Microsoft is in real trouble to say the least. MORE ACCURATE DETAILS of what happened in the first place to the site and who coded it would help here Slashdot!
This message was not sent from an iPhone because Peter Sellers really was a deviated prevert without a dime for the call
I can understand Google and Redhat... but Oracle? Talk about having a fox in the hen-house.
Enlisting JUST ONE of the tech giants would be more productive.
You can't fix stupid. End it, don't mend it!
an ill wind that blows no good
And Elon Musk hasn't even been MENTIONED yet?!
He already designed a solution based on "a series of tubes", but it was dismissed as being impractical in the real world because it didn't involve enough contractors for implementation.
Why does everyone in the country need to use the website at once? Couldn't the problem be fixed with a little javascript function:
1. Enter your Social Security Number
2. Based on your Social Security number, your enrollment date is 1-Nov-2013 - 7-Nov-2013 or anytime after 31-Dec-2013. If you do not know or do not have an SSN, your enrollment date is after 15-Jan-2014. Click here to have an email reminder sent on your enrollment date.
They could instantly cut the website demand by 90% by dividing enrollments up by the last digit of the SSN of the primary enrollee.
Crap, now the NSA will have a backdoor into the government!
And we thought it was expensive and past deadline NOW.
Mod me down with all of your hatred and your journey towards the dark side will be complete!
Instead of fixing a bunch of hopeless code, why can't they start over the damn thing - with a properly designed paradigm ?
Muchas Gracias, Señor Edward Snowden !
Google will spend all of their time working on tracking users, Oracle will insist on integrating dozens of Oracle products costing hundreds of millions of dollars, and RedHat will rewrite the system while removing the capability of running it over a network.
But in all seriousness, the reason this web site is in shambles is because the developers weren't given nearly enough time to implement a product this complex. And if years of development wasn't enough time, the government thinks that a few big tech companies can fix the problem in a single month? Even the best engineers will require weeks to understand how the system currently works, several more weeks coming up with a plan to fix what's broken, and months to implement the solution. This just goes to show how drastically people underestimate the complexity of software development. I wish those engineers the best of luck - they're being set up for failure.
Alright, who is getting crucified over this one?
My guess would be that healthcare.gov runs on top of an Oracle DB already, so Oracle probably has a few engineers that can be brought in to help identify and restructure problematic queries and/or tweak server settings to eek out a bit more performance. It's also a distinct possibility that the back end is in Java, so Oracle has a few knowledgeable Java engineers too.
Oracle's one-word answer for the DB -- Exadata...
Karma: Bad
It's not 600 million.
http://www.washingtonpost.com/blogs/fact-checker/wp/2013/10/24/how-much-did-healthcare-gov-cost/?wprss=rss_politics
Webserver? Middleware? Database? Hardware??
Karma: Bad
No, apparently they already had experienced PayPal's website.
#DeleteChrome
No way you're dropping analysts in to fix a problem in a couple weeks that's probably going to be done in a month anyway.
I swear to God...I swear to God! That is NOT how you treat your human!
As per article it is AT LEAST $174 million plus a lot of other stuff that's hard to account for. Still seems ridiculously high.
Negative moral value of force outweighs the positive value of good intentions.
First Lady Michelle Obama and her Princeton classmate whose company received the no-bid government contract to build the HealthCare.gov Obamacare website were both members of a black student organization that caused a tense scene on campus by inviting a PLO leader who advocated for terrorism.
Michelle Obama ’85 and her classmate Toni Townes-Whitley ’85, a senior vice president at CGI Federal, were both students at the university when their groups the Organization of Black Unity (OBU) and the Third World Center (TWC) engaged in a confrontation with Jewish students on campus.
Source Daily Caller: http://dailycaller.com/2013/10/28/michelle-obama-and-cgi-federal-executive-belonged-to-student-group-at-princeton-that-hosted-pro-terrorist-speaker/#ixzz2jM3P01QH
Guess what S in NSDAP stood for.
Negative moral value of force outweighs the positive value of good intentions.
I think they are going to be prayin' . . . real good!
North Korea has Democratic and Republic in its official name, this does not implies that it is a democractic republic and it does not implies democracy and republic are totalitarism.
Same fro NSDAP: the fact that someone grabbed and kinked a concept does not invalidates it universally. And we we talk about "social" in the US, it has nothing to do with soviet Russia.
calvary != cavalry.
One is the hill where Jesus is believed to have been crucified.
One is soldiers riding horses into combat.
-Styopa
... It Means What You Think It Means
Calvary: the site immediately outside Jerusalem's walls where Jesus was crucified.
Cavalry: soldiers or warriors who fought mounted on horseback.
Thanks to file sharing, I purchase more CDs
Thanks to the RIAA, I buy them used...
Apparently, there are two problems with the deployment process (not the health care logic behind it)... First was that the code didn't scale, second is that it was not tested in a meaningful way to show it. Let me guess, it was built by the cheapest contractor and we assumed their leadership and subject matter expertise would guarantee success? Sounds like the contract clause that says payment is based on performance should be exercised. There is one right? WHy should we pay CGI for their work? Time for them to shoulder the pain.
Time for a new Political party in the US (or two!) One is off the rails Other cant pony up a leader.
Or that Oracle already built a failed exchange website in Oregon.
At the same time, it's kind of entertaining to watch the general public start to grapple and become aware of the same project management issues I've had to deal with for the last decade.
"First they came for the slanderers and i said nothing."
I have. It's not that bad. Really.
Now I don't need insurance as I already have it from my employer, but I was curious how bad the site was. But it didn't turn out being difficult or error prone at all to sign up. It took about 15 minutes total and I had the eligibility report for me and my daughter. Some nit picks:
1) The confirmation email was one of three emails i got from healthcare.gov when signing up. That could confuse some people.
2) One required field on one page was scrolled off the bottom, and no scroll bar appeared to indicate that. Mouse wheel scrolling down solved that, but if there are many pages with that problem it could be confusing.
That's about it. Maybe I just lucked out, bit it was an easy site to use.
Why didn't they bring in someone like Amazon who is good at scaling interactive websites instead of static query serving like Google?
Is the site built using Oracle? If not, what do they expect Oracle to do to help?
Most importantly: Why weren't "hired guns" like this brought in to do the design and architecture in the first place? WTF were they doing using a provider who is already involved in a failing website serving a much smaller community?
As with most large scale government projects, the whole thing just smacks of mismanagement and scope creep.
The only way things could have gone worse is if they'd hired Accenture. Then the price would have doubled for the same crap.
I do not fail; I succeed at finding out what does not work.
nice to see the us gov making friends with the big tech they supposedly just made mad. lets all be friends, share the data, and loot the taxpayers. no need to for anyone to feel left out. ok, need to get ms, yahoo and then some onboard. next.
Oracle's online systems are as dinosaur as the company. Taking literally 30 seconds to "load" a web page... Take them out the back and finish them off.
Get nine women!
"False hope is why we'll never run out of natural resources!" - Lewis Black
I think a big part of any near future fix for the website will be a massive cull of features and functionality. For example, a number of features, perhaps even the entire website, could be handled by a human operator. They could have a few tens of thousands of "trained" operators in makeshift offices with the necessary telecommunication equipment and liability protection inside of a month, should they chose to go that route.
So I could see, for example, the entire website reduced to a shell which might collect some user data or discuss health insurance options with the eventual transaction handled over the phone.
So to go with the common analogy around here, instead of one woman having a baby in nine months, they'll have nine women hold babies for a month and leave the big fixes for next year or later.
It all depends on the quality of the existing code base. More often than not, it's better to start from scratch.
No, no, no, NO it's not. Do your research. Starting from scratch will end up taking you just as much time. Refactor, clean up and fix.
Red Hat will have everything switched to their version of linux, Oracle will have everything switched over to their db, and Google will data-mine it all.
to make shitloads of money. Muaha. Muahaha. Muahahahahahahha
oh man this is gonna be good (sits back with a bowl of popcorn)
What makes me kind of sad is that most companies and governments never learn from lessons that the past have already taught us. It was obvious that connecting so many different vendors through one web site was going to be a pain unless some communication protocol was established as a standard. What if this sort of thing, after being established, was thoroughly tested before a universal front end was created? However obvious this was, it didn't happen for the usual reasons; politics, delegation of blame, responsibilities and the usual unclear communication.
(grumpy old man voice) If we were ate war and the military was responsible for this, they would have gotten this sorted out in 24 hours! If only lives would depend on it, things would have gone so much smoother! Oh wait....
I was promised a flying car. Where is my flying car?
Maybe that 1% of time Oracle DB is the right solution, and then you need it and you need to pay DBAs and consultants to do the tuning.
But in other cases, Oracle is an overkill. And Postgres with minimal effort will Just Work, while just the effort to set up Oracle and keep it running is going to be expensive (DBAs, consultants) and horrible. Especially if you can spend all the money you saved on licences on proper hardware. And I have experienced more bugs in Oracle than in Postgres, both in weird query optimizer behaviour, and outright incorrect data handling. And there is no way to get them fixed, at least in timely manner, not even with most expensive support options.
And don't get me started on Oracle enterprise applications- they are horrible. And given the history of company of overcharging their customers and delivering crap, I wouldn't trust them with any project.
About the only thing produced by Oracle that I enjoy is Java. And I still shudder every time I think that it is owned by Oracle, although they seem to be doing a reasonably good job getting new versions out of the door.
--Coder
I do agree with a lot of points listed here. Rolling your own development team is not easy, nor cheap.
However, when you hire companies to do contracts this big, you end up with horrible horrible mess. First, you get to hide the bigest contractors available (because the project is big), like Accenture or similar. And they are the worst, and have the worst (cheapest) people available working for them. The only selling point they have is the headcount, not quality of developers. And they will assemble a new team for this project, from random guys, with flaky qualifications and skills, and often no prior experience.
And all those companies are for-profit. Which means they will find ways to overcharge you by A LOT. Like 10x-20x the cost of a good sized team for several years.
In the end, a project like this that has been outsourced to big IT companies is almost guaranteed to cost you a fortune, and end up a failure.
Now it's possible (and likely) to end up with a failed project with an in-house team, but choosing in-house development vs contracting is not as clear cut as you seem to suggest.
--Coder
Thanks for that link, a lot of screwy numbers floating about. So, currently around $170 million, with an "upwards potential of $300 million."
It's not 600 million.
http://www.washingtonpost.com/blogs/fact-checker/wp/2013/10/24/how-much-did-healthcare-gov-cost/?wprss=rss_politics
and I quote from your precious article:
"Update, Oct. 30: In testimony on Capitol Hill, Health and Human Services Secretary Kathleen Sebelius said, in response to a direct question: “Congresswoman, we have spent about $118 million on the website itself, and about $56 million has been expended on other IT to support the web.”
That adds up to $174 million."
Do you really think this PIECE OF GARBAGE , was worth 174 MILLION US DOLLARS?.... REALLY? any way hippie liberals want to spin this, you BLEW 174 MILLION on GARBAGE.
What software architect here thinks they can do this entire site for substantially less? keep in mind, these are the OPTIMISTIC, hippie liberal numbers as reported by the washingtonpost.com . Yes, I'm not a democrat. and before you denounce me, look at the INSANE numbers here, and realize they probably underestimated it.
The truth lies somewhere between their LOW ESTIMATE, and the republican HIGH ESTIMATE.
It's 'Cavalry', not 'Calvary', although Healthcare.gov may end up crucified by these 'Tech Giant's'... Personally, I would have used Amazon and H-P or IBM , definitely NOT ORACLE.... $$$$$$
President Obama has spent SIGNIFICANTLY MORE than ANY OTHER PRESIDENT.
You're comparing apples to oranges. President Obama had significantly lower "increase" in spending from the prior year before. Largely, because he came in afte the super-massive emergency trillion $+ banking bailout. That was a) a huge expenditure b) an emergency expenditure. What President Obama did was continue at the spending level every year since.
In other words, imagine your household expenditure is $50K. But you had a major illness requiring a $50,000 hospital surgery. So that year, your expenditure was $100K. The next year you spend $98K and exclaim you've reduced spending. But what you're not admitting is that you're factoring an emergency expenditure into the regular budget. Then later spending that same amount without an emergency expenditure.
Bush Term 1
$8.9 Expenditure
$1.2 Deficit
Bush Term 2
$11.8 Expenditure
$2.2 Deficit
Obama Term 1
$11.9 Expenditure
$2.3 Deficit
Here is the key, ALL the deficits under President Bush were below $500 billion until 2009. 1/2 were around the $150-$300 billion. Three were around the $400-$450 billion. Then in 2009 we get a $1.4 trillion deficit thanks to the bank bailouts.
The deficits for every year of President Obama's 1st term exceed $1 trillion. While none were as high as the $1.4 during the banking crisis. Two were $1.3 trillion, and two were $1 trillion.
In other words, President Obama took an emergency situation that equated to an additional $1 trillion in deficit spending and made it standard. So yes, President Obama has "increased" spending less. But he has maintained "emergency" spending levels for every year of office.
Wouldn't that kind of be like putting Monsanto execs on the FDA? Oh... right.
Calvary, which is also called Golgotha, is a hill.
Cavalry is a horse or highly mobile military unit.
Rick Ungar = STUPID
"UhObama was never a Congressman. He did serve as a Senator."
Do you want to put any weight into an article written by such an ignorant man?
"The United States Congress is the bicameral legislature of the federal government of the United States consisting of two houses: the House of Representatives and the Senate."
- Wikipedia
http://www.forbes.com/sites/rickungar/2012/05/24/who-is-the-smallest-government-spender-since-eisenhower-would-you-believe-its-barack-obama/
***
Okay, what this article is really saying is that President Obama took office after the largest emergency budget expenditure in U.S. history. One that pushed the deficit up from President Bush's average of $305 billion to over $1.4 trillion. Largely due to a 1 time $1 trillion dollar bailout of the U.S. and global banking system.
Then he lauds President Obama for only having an average of $1.2 trillion dollar deficit. Exclaiming, see...see... President Obama lowered the deficit from $1.4. Reducing the deficit by almost 15%.
What Mr. Ungar fails to point out, is that President Obama kept spending at emergency levels. Take out that one time emergency expenditure and compare the average deficits.
What you see is President Bush's $305 billion versus President Obama's $1,225 billion a year deficit.
So how much did President Obama increase the deficit spending? 400%
Even factoring the 2009 fiscal year with the bank bailout. President Bush's average deficit was $443 billion vs $1,163 billion (and that's assuming the 2013 ends with the smaller estimated $0.9 trillion deficit - have my doubts).
Even then, it equates to a 2.5x increase in deficit spending over President Bush.
And, remember that half a trillion of that 2009 budget was the emergency stimulus bill that President Obama passed.
So it seems that Healthcare.gov's "data services hub" is all built with Oracle, using RHEL app servers? Other reports mentioning them bring in Amazon and Google people as well...I am guessing for the cloud scaleability part. Funny that some nontraditional private sector companies are getting the lucrative contracting gigs. Usually it goes to companies whose revenue is 90+ percent from the government.
So we have three more no-bid contracts for an undisclosed sum.
Of course not even a whimper of discontent from the media, the lefties, or the commentators here, many of whom wailed and howled for years about such practices when the evil Rethugnikans did this out of necessity for ONE MONTH. Because as we all know some pigs are equal, but some pigs are not as equal as others.
By the time this is over we'll have a trillion dollar website. And then we will hear about the CRISIS where the whole thing has to be rewritten, or we are all doomed, and if the opposition party protests, well, they hate children, want grandma to starve, and just want people to die.
Murphy was an optimist
I was under the (possibly misinformed) impression that they hired ONE contractor who immediately turned it around and created 55 subcontracts out of it (which is pretty typical for govt work)
Ok, this is just begging for the Hitler ranting video meme for Obamacare... :)
Somehow I have to think there was a little more "pressure" from the Nazi's to not fail.
To quote DarthVader "You have failed me for the last time" *force choke*
That also needs to be made into a new internet meme... Perhaps after it fails the end of November...