Slashdot Mirror
How Healthcare.gov Changed the Software Testing Conversation
An anonymous reader notes an article about how the tribulations of Healthcare.gov brought the idea of software testing into the public consciousness in a more detailed way than ever before. Quoting:
"Suddenly, Americans are sitting at their kitchen tables – in suburbs, in cities, on farms – and talking about quality issues with a website. The average American was given nightly tutorials on load testing and performance bottlenecks when the site first launched, then crumbled moments later. We talked about whether the requirements were well-defined and the project schedule reasonably laid out. We talked about who owns the decision to launch and whether they were keeping appropriate track of milestones and iterations. ... When the media went from talking about the issues in the website to the process used to build the website was when things really got interesting. This is when software testers stepped out of the cube farm behind the coffee station and into the public limelight. Who were these people – and were they incompetent or mistreated? Did the project leaders not allocate enough time for testing? Did they allocate time for testing but not time to react to the testing outcome? Did the testers run inadequate tests? Were there not enough testers? Did they not speak up about the issues? If they did, were they not forceful enough?"
Ya those damn testers, they just can't communicate the issues to management. Like that NASA engineer and the O-rings. Stop blaming the testers.
It is not really a question of testing. Parts of the software were missing or incomplete. You can't test what isn't there.
These same questions plague Battlefield 4
It does not sound to me as though known management tools were used. Did they sit down with the government personnel in charge, and present their approach, and what the site would look like (menus, flow, etc) when finished? Were there testable milestones, and a final presentation of working software? It sure doesn't sound like it.
I cannot imagine a worse job than to have worked on that project.. The ratio of "status update" meetings and management pud-pulls to useful work accomplished must have been damn close to infinity..
There are over 2.8 million words of Obamacare regulations!
I challenge anyone to create a website that conforms to such a huge number of rules -- some of them probably contradictory!
Conservatives constantly point out how excessive regulation makes doing business difficult. Well it makes things difficult on the government, too. Let's be fair.
This is just yet another big government project gone awry. We get these in the UK all the time. I seriously doubt anyone is talking about the testing of this particular project though. Those involved in testing will just keep doing what they do, good ones doing it properly, bad ones doing it, well badly. The other 99% of the population will just bitch about the site as being generally crap but they won't be saying 'They really should have done more integration and load testing'
I want a list of atrocities done in your name - Recoil
load testing and performance bottlenecks"
That's great but how about we teach the average American how to spot Europe on a map first.
Most of the Affordable Care Act has nothing to do with the web site. The site didn't have to implement those "2.8 million words of Obamacare regulations" as code: it only had to match patients up with insurance plans, which means interacting with dozens (hundreds?) of government and industry databases.
Some states, like California, managed to implement their sites without any of the problems of the federal exchange. The federal exchange mainly suffered from (1) being rushed, and (2) having to deal with a larger number of external systems than any single state exchange.
Visual IRC: Fast. Powerful. Free.
Performance and scaling should have been addressed in the design phase
Comment removed based on user account deletion
Give the QA guys a video camera! This charade won't last!
Why? Is Europe's location somehow significant to average Americans?
No the project leads gave plenty of time for testing, development, and even kept the WH up to date on what was happening. What happened though was the Obama administration pushed through something that wasn't ready, and wouldn't be ready for long past it's actual inception date. And this of course is because the administration sat on it's backside for an extended period of time, then waved their hands and said a couple of years should be more than enough.
The committee meetings are chock full of very useful information on this, lots of waffle, but surprising bits in the waffle itself. And most of it revolves around, but we..and..they said...followed by...we were going to do it anyway, but it's not our fault we pushed it out early.
Om, nomnomnom...
The problem underlying the entire fiasco — and the less-impacting others like it (Amtrak, anyone?) — is that whatever the government does, is done poorly .
"Not bad for a government job," — is part of vernacular, yet, a curious mix of well-meaning idealists and self-serving demagogues manage to convince the public to try again every once in a while...
In Soviet Washington the swamp drains you.
Imagine being the QA inspector on a 1985 Jugo car. No matter what you say, the entire thing is a POS. The only question is whether you need your paycheck that badly. Politics and unrestrained corruption simply don't mix well with code.
One of the most insightful truths ever told to me:
It is always management's fault.
This goes right to the root of the tree, because by definition if the people further out couldn't get the job done or didn't have the right resources to do it, it was management's responsibility to fix those problems. The buck stops with the most senior managers on a project, whose only two choices are to explain what is needed to succeed and then do so if given those things, or to fail.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
Americans theoretically are supposed to approve of their overseas combat operations, so it might be better to have them identify Iraq and Afghanistan, but the percentage of positive responses would be so depressingly low, finding Europe would be a reasonable consolation prize for the poll-takers.
Why? Is Europe's location somehow significant to average Americans?
And there is the proof of the OP's implied statement
That's great but how about we teach the average American how to spot Europe on a map first.
I think finding Europe is the least of their problems: The Chaser: War on Everything - Americans
(Yes I know it was probably all in the editing .. still .. you've got to be elfin joking)
I am Slashdot. Are you Slashdot as well?
SQA as a red-headed stepchild has been an issue for many, many years. It's just that most troubles/failed software systems don't have the widespread public exposure that Healthcare.gov has; even the most brain-dead corporation would not have launched such an incomplete and bug-ridden system to a vast end-user bases.
Some years ago, I led a review of a late (4 yrs vs 2 yrs estimated) and very over-budget ($500M vs. $180M estimated) corporate software project. The core problems had everything to do with SQA, starting with the fact that there was no SQA organization; all testing was done on an ad hoc basis by individual teams and organizations. Adding to that problem was the fact that there was no coherent architecture. After 4 years and $500M, there were no systems that were ready to go into production. Far too common in industry and especially in government. ..bruce..
Bruce F. Webster (brucefwebster.com)
A Swiss friend of mine visited DC. I live in So Mo. We chatted on the phone and he suggested we get together for lunch the next day. So, I agreed and told him the city in Virgina we could meet in after driving eleven hours.
They're no better with our geo than we are with theirs.
If you find out in testing that your architecture or design does not cut it, you are screwed. The only thing you can usually do is scrap the project and start again. Testing does only work for simple things like simple busiess logic and the like, where you know the characteristics very well beforehand. For anything that is a new design, the only thing that helps is very capable and experienced architects and designers that have a good change to get it right by intuition. This will be people that can do architecture, design and implementation and can do all three well. Not many of those exist, but there is no replacement for them. Those that think they can do things on the cheap without not only having this type of expert but also listening to them closely will fail. This can be observerd time and again and can alost be called a "well established industrial practice", because quite a few "managers" do not actually know that it can be done better. Funny thing, in other fields, you have chief enineers, architects and the like and the critical work is not given to people that are likely to fail. Only IT messess it up regularly, because talent and exerence is not respected.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
The little awareness they glean of the craft from this incident will just as likely be used to their detriment as their advantage. quote quote...something about a little bit of information...
Happiness in intelligent people is the rarest thing I know.
Ernest Hemingway
I don't get it. Failing to understand why JoeyRox assumes Europe's location is signifcant to average Americans is very differnt from being ignorant of its location.
What point are you trying to make?
The state exchanges also were planned solely by and for an individual state. Originally, the federal website was supposed to be used by 20 or so states. It's now being used by 36. The original scope of the project may well have worked for half as many states. It seems like it was never properly scaled and instead, a solution designed to accommodate a certain number of users was shoehorned to accommodate double that initial number.
Yes. The average American is of recent European descent, speaks a European language, and lives under a system derived from European thought history.
You've got to ask yourself, who's building and deploying reliable, performant, extremely scaled apps these days? Who has been doing that successfully for over a decade? Why don't we ask them to build our big app? Or if they're busy, ask them who they would recommend.
Where are we going and why are we in a handbasket?
Ok, so some projects, as has been pointed out, are doomed from the very first bad architectural decision (or lack of architectural decision.)
But regardless of that factor, the most common thing I've seen is management/corporate promising a particular release date, in a contract, say, and eventually getting around to telling development/engineering, who say, if they're brave, um, that's not possible. If they are less brave, they smile and get on with faking it, all the while knowing it's impossible in the time given. If they're highly skilled and properly whipped, they'll get something that looks superficially ok out the door on schedule, but don't ever, ever, try to use it.
Where are we going and why are we in a handbasket?
That's great but how about we teach the average American how to spot Europe on a map first.
That's great but how about we teach the average American how to spot the USA on a map first.
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
I'm a developer that has been working as a tester in the last 5 years.
Always coding to run performance / load / whatever not manual testing.
In my experience working at bank or big publishers, my main problem is always budget.
Enough money for everything except the right number of people testing, the right tools or infrastructure for testing, and so on.
I don't expect that was different with healthcare.gov.
By all accounts, there was some testing done, however inadequate. Thing is, the system utterly failed that testing and THEY DEPLOYED IT ANYWAY. If you're going to ignore the testing, why even do it? Just throw open the doors and hope for the best. Which is what they did, apparently.
Democrats wanted it, so it was doomed to fail since nothing they do work. Obama was told it was not ready, but he ignored it. You can't really blame the programers or testers, there simply was not enough time to fully develop the website. So it comes down to Obama to be blamed for the website problems.
But it could have been Obama's plan for it to fail at the start so he can delay a lot of Obamacare till after midterms. That plan was not very successful though, since people are starting to see how horrible Obamacare is, and that Obama knowingly lied about keeping their healthcare, doctor and STILL have to pay more then their old plan.
The only defense I hear from democrat voters are "The healthy should have to pay unhealthy's medical bills, because its unfair that you are healthy."
Because otherwise they look like idiots to th rest of the world
hah, I've been doing software testing for 25 years. :p
"changed the conversation"
"owns the decision"
What kind of meaningless hipster newspeak is this?
If hipster horseshit is representative of the methods used to build Healthcare.gov then no one should be surprised that it failed.
http://dictionary.reference.com/browse/snafu+principle
In the beginning was the plan, and then the specification; And the plan was without form, and the specification was void. ...
And darkness was on the faces of the implementors thereof; And they spake unto their leader, saying: "It is a crock of shit, and smells as of a sewer."
It happens occasionally here in the Southwest U.S. that European tourists forget they can drive for eleven hours and not see anyone, or anything. Including a petrol station.
...to do what you just did.
It's a left-wing meme to claim that conservatives refuse to change their minds based on facts, but the FACT is that you have presented no "facts" that would cause a rational person to change his views. Obamacare is just the latest proof that both libertarians AND conservatives are right about big government..... even when run by people who LOVE big government (and who we are told are the smartest and best-qualified to run it), the federal government can spend obscene amounts of money trying to do something the private sector regularly does and totally fail
VA Medical care is AWFUL compared to what's available in the private sector. Sure, they MAY be better than the socialized healthcare in some other countries, BUT the VA system is a mess compared to the private sector. Presidents and former Presidents get healthcare via the military system ("commander in chief" ring a bell?) but that has nothing to do with quality; they'd get the best of what's available no matter what system they are in. The fact that the military medical folks deal best with INITIAL care of battlefield injuries is simply because they face those injuries more than civilian docs at your local hospital. Please refrain from ranting about the wonders of military medical care unless you have endured it. None of these healthcare-related points you tried to make show that the federal government does things better than anybody else.
Yes, NASA managed to put a man on the moon..... there were some bright people running the place in the 60's who contracted with companies like North American Aviation, Grumman, Chrysler, MIT, etc and these non-government corporations actually designed and built the systems. NASA ran the thing and managed/coordinated the effort, but did NOT do all the work. Look at the old pictures and films of those times and notice all the technicians with lab coats with various corporate logos; read the documents and see all the back-and-forth between the government managers and the corporate implementers - Obama SHOULD have done that before attempting "Obamacare", he might have learned a few valuable lessons. NASA and the contractors made many missteps on the way to the moon landing success including miscommunication and poor oversight which lead to the Apollo 1 fire and dead astronauts. NASA overcame this by spending lots of money and changing the way they and their many contractors worked and interacted. The huge levels of obscene spending required to succeed were not sustainable, so only a dozen men walked on the moon before the program was terminated. Ever noticed how many post-Apollo NASA projects have been started, spent lots of money, and then been cancelled without every flying??? Ever noticed how many military projects get started in the US and then go way over budget and under-deliver on what was promised?
The thing the moon landing had in common with the Normandy invasion is this: The federal government can indeed achieve an occasional great thing by throwing huge numbers of people and massive piles of cash at a problem until it's "solved"; when money is no object, even the most incompetent and clumbsy outfit can achieve a goal BUT very few projects get enough public support for those levels of spending and when they DO the support only lasts a few years. Conservatives "get" this which is why they support military spending generally while complaining about government waste and inefficiency (liberals always misunderstand/mis-portray this as hypocrisy but it's just realism). We want as little government (which is wasteful and inefficient) as possible, but recognize that the military has a job that cannot reasonably be done in any other way (no, mercenaries are generally not a good idea), so the waste and inefficiency of the pentagon is a necessary evil (though most of us would like to see FAR greater oversight of it). When there is a viable alternative to govt, it should be used.
Might have helped to actually open the project to bids to all companies and not just to ONE crony of Michelle Obama's and a company who had a history of already failing TWICE at similar (and smaller) projects.
The FIRST bug was incompetance/negligence from the very top.
With these dirtbag politicos and a co-conspirator press we wont be finding out any detials, just vague excuses that it WASNT the administrations fault.
Sorry, you dont achieve such things by talking about them and just assuming it will get done.
BTW - Ive made a good living doing software testing (and programming) for decades and have seen screw ups (but none as big as this cluster-obama).
Crazy to go on about QA. The functional specs were, by necessity, incredibly convoluted and they kept changing for months after the project started. Even the best testers in the software world cannot qualify a half done system.
https://en.m.wikipedia.org/wiki/Test-driven_development is better for Govt projects
Casteism
Testing could not start because the government continuously changed the specs, even a few hours before public release fundamental code was changed.
What's "So Mo"? Southern Missouri? Technically that would be "So MO" to disambiguate from Montana...
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
Why? Is Europe's location somehow significant to average Americans?
Well you know, you might like to bomb us sometime. Then it comes in handy to know where it is!
Stefan Axelsson