Slashdot Mirror
How Healthcare.gov Changed the Software Testing Conversation
An anonymous reader notes an article about how the tribulations of Healthcare.gov brought the idea of software testing into the public consciousness in a more detailed way than ever before. Quoting:
"Suddenly, Americans are sitting at their kitchen tables – in suburbs, in cities, on farms – and talking about quality issues with a website. The average American was given nightly tutorials on load testing and performance bottlenecks when the site first launched, then crumbled moments later. We talked about whether the requirements were well-defined and the project schedule reasonably laid out. We talked about who owns the decision to launch and whether they were keeping appropriate track of milestones and iterations. ... When the media went from talking about the issues in the website to the process used to build the website was when things really got interesting. This is when software testers stepped out of the cube farm behind the coffee station and into the public limelight. Who were these people – and were they incompetent or mistreated? Did the project leaders not allocate enough time for testing? Did they allocate time for testing but not time to react to the testing outcome? Did the testers run inadequate tests? Were there not enough testers? Did they not speak up about the issues? If they did, were they not forceful enough?"
Ya those damn testers, they just can't communicate the issues to management. Like that NASA engineer and the O-rings. Stop blaming the testers.
It is not really a question of testing. Parts of the software were missing or incomplete. You can't test what isn't there.
These same questions plague Battlefield 4
It does not sound to me as though known management tools were used. Did they sit down with the government personnel in charge, and present their approach, and what the site would look like (menus, flow, etc) when finished? Were there testable milestones, and a final presentation of working software? It sure doesn't sound like it.
This is just yet another big government project gone awry. We get these in the UK all the time. I seriously doubt anyone is talking about the testing of this particular project though. Those involved in testing will just keep doing what they do, good ones doing it properly, bad ones doing it, well badly. The other 99% of the population will just bitch about the site as being generally crap but they won't be saying 'They really should have done more integration and load testing'
I want a list of atrocities done in your name - Recoil
I cannot imagine a worse job than to have worked on that project.. The ratio of "status update" meetings and management pud-pulls to useful work accomplished must have been damn close to infinity..
You haven't worked on my project.
Faster! Faster! Faster would be better!
load testing and performance bottlenecks"
That's great but how about we teach the average American how to spot Europe on a map first.
Most of the Affordable Care Act has nothing to do with the web site. The site didn't have to implement those "2.8 million words of Obamacare regulations" as code: it only had to match patients up with insurance plans, which means interacting with dozens (hundreds?) of government and industry databases.
Some states, like California, managed to implement their sites without any of the problems of the federal exchange. The federal exchange mainly suffered from (1) being rushed, and (2) having to deal with a larger number of external systems than any single state exchange.
Visual IRC: Fast. Powerful. Free.
Performance and scaling should have been addressed in the design phase
Comment removed based on user account deletion
Why? Is Europe's location somehow significant to average Americans?
No the project leads gave plenty of time for testing, development, and even kept the WH up to date on what was happening. What happened though was the Obama administration pushed through something that wasn't ready, and wouldn't be ready for long past it's actual inception date. And this of course is because the administration sat on it's backside for an extended period of time, then waved their hands and said a couple of years should be more than enough.
The committee meetings are chock full of very useful information on this, lots of waffle, but surprising bits in the waffle itself. And most of it revolves around, but we..and..they said...followed by...we were going to do it anyway, but it's not our fault we pushed it out early.
Om, nomnomnom...
> I challenge anyone to create a website that conforms to such a huge number of rules -- some of them probably contradictory!
Sounds like any other regulatory burden. Are the things at Amazon FCC approved? Are they UL listed? Do they pass muster by the USDA?
All of that stuff is outside the scope of the website and it should be the case for Obamacare too.
Your kind of thinking is why it was such a disaster and why 3 guys managed to throw together a window shopping frontend with little effort.
A Pirate and a Puritan look the same on a balance sheet.
Imagine being the QA inspector on a 1985 Jugo car. No matter what you say, the entire thing is a POS. The only question is whether you need your paycheck that badly. Politics and unrestrained corruption simply don't mix well with code.
One of the most insightful truths ever told to me:
It is always management's fault.
This goes right to the root of the tree, because by definition if the people further out couldn't get the job done or didn't have the right resources to do it, it was management's responsibility to fix those problems. The buck stops with the most senior managers on a project, whose only two choices are to explain what is needed to succeed and then do so if given those things, or to fail.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
Why? Is Europe's location somehow significant to average Americans?
And there is the proof of the OP's implied statement
That's great but how about we teach the average American how to spot Europe on a map first.
I think finding Europe is the least of their problems: The Chaser: War on Everything - Americans
(Yes I know it was probably all in the editing .. still .. you've got to be elfin joking)
I am Slashdot. Are you Slashdot as well?
Yes, yes, because every bridge the government builds falls down three or four times a day in the first couple of weeks after it's opened.
Slashdot - News for Nerds, Stuff that Matters, in ISO-8859-1 Has just realised that beta makes this signature redundant
Conservatives constantly point out how excessive regulation makes doing business difficult. Well it makes things difficult on the government, too. Let's be fair.
Isn't that rather like saying "Ladies and gentlemen of the jury, have compassion on my client. By killing his parents he became an orphan."?
SQA as a red-headed stepchild has been an issue for many, many years. It's just that most troubles/failed software systems don't have the widespread public exposure that Healthcare.gov has; even the most brain-dead corporation would not have launched such an incomplete and bug-ridden system to a vast end-user bases.
Some years ago, I led a review of a late (4 yrs vs 2 yrs estimated) and very over-budget ($500M vs. $180M estimated) corporate software project. The core problems had everything to do with SQA, starting with the fact that there was no SQA organization; all testing was done on an ad hoc basis by individual teams and organizations. Adding to that problem was the fact that there was no coherent architecture. After 4 years and $500M, there were no systems that were ready to go into production. Far too common in industry and especially in government. ..bruce..
Bruce F. Webster (brucefwebster.com)
A Swiss friend of mine visited DC. I live in So Mo. We chatted on the phone and he suggested we get together for lunch the next day. So, I agreed and told him the city in Virgina we could meet in after driving eleven hours.
They're no better with our geo than we are with theirs.
If you find out in testing that your architecture or design does not cut it, you are screwed. The only thing you can usually do is scrap the project and start again. Testing does only work for simple things like simple busiess logic and the like, where you know the characteristics very well beforehand. For anything that is a new design, the only thing that helps is very capable and experienced architects and designers that have a good change to get it right by intuition. This will be people that can do architecture, design and implementation and can do all three well. Not many of those exist, but there is no replacement for them. Those that think they can do things on the cheap without not only having this type of expert but also listening to them closely will fail. This can be observerd time and again and can alost be called a "well established industrial practice", because quite a few "managers" do not actually know that it can be done better. Funny thing, in other fields, you have chief enineers, architects and the like and the critical work is not given to people that are likely to fail. Only IT messess it up regularly, because talent and exerence is not respected.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
The little awareness they glean of the craft from this incident will just as likely be used to their detriment as their advantage. quote quote...something about a little bit of information...
Happiness in intelligent people is the rarest thing I know.
Ernest Hemingway
The problem underlying the entire fiasco — and the less-impacting others like it (Amtrak, anyone?) — is that whatever the government does, is done poorly .
I realize that that's a right-wing meme, and it's rare for conservatives to change their minds based on the facts, but it's not true.
The military and Veterans Affairs medical centers give some of the best care in the world. I've read the studies that compare them to other centers around the world. They've got the data.
Ronald Reagan got his colon and prostate surgery at Walter Reed. Watch what they do, not what they say.
If you got a head injury in Iraq, you'd have the best chance in the world of surviving with as much of your brain left as possible in the military health system. Ditto with saving a leg or an arm.
The National Institutes of Health is the biggest medical research center in the world. They've done more important research, and won more Nobel prizes, than the entire U.S. pharmaceutical industry put together.
I leave it to Gordon Crovitz to explain how the U.S. government created the Internet.
NASA put the first man on the moon.
Does the invasion of Normandy count?
Yes. The average American is of recent European descent, speaks a European language, and lives under a system derived from European thought history.
You've got to ask yourself, who's building and deploying reliable, performant, extremely scaled apps these days? Who has been doing that successfully for over a decade? Why don't we ask them to build our big app? Or if they're busy, ask them who they would recommend.
Where are we going and why are we in a handbasket?
Ok, so some projects, as has been pointed out, are doomed from the very first bad architectural decision (or lack of architectural decision.)
But regardless of that factor, the most common thing I've seen is management/corporate promising a particular release date, in a contract, say, and eventually getting around to telling development/engineering, who say, if they're brave, um, that's not possible. If they are less brave, they smile and get on with faking it, all the while knowing it's impossible in the time given. If they're highly skilled and properly whipped, they'll get something that looks superficially ok out the door on schedule, but don't ever, ever, try to use it.
Where are we going and why are we in a handbasket?
That's great but how about we teach the average American how to spot Europe on a map first.
That's great but how about we teach the average American how to spot the USA on a map first.
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
I'm a developer that has been working as a tester in the last 5 years.
Always coding to run performance / load / whatever not manual testing.
In my experience working at bank or big publishers, my main problem is always budget.
Enough money for everything except the right number of people testing, the right tools or infrastructure for testing, and so on.
I don't expect that was different with healthcare.gov.
By all accounts, there was some testing done, however inadequate. Thing is, the system utterly failed that testing and THEY DEPLOYED IT ANYWAY. If you're going to ignore the testing, why even do it? Just throw open the doors and hope for the best. Which is what they did, apparently.
You don't know much about Amtrak, do you? Amtrak is the result of rail companies no longer wanting to deal with the passenger business they had left after the rise of the automobile. So they got together and convinced the government to take over that side of things, while leaving them free to engage in the business they wanted, freight.
Which they do very nicely.
But passengers? They don't want the bother, they don't want the trouble, they'd rather leave it to highways (run by the government, in case you didn't know) or the airlines (and fuck you if you've never noticed how poorly that deregulated shit is doing), even the government doesn't want the job, except in select areas where it works very well.
And you know what's a common part of the vernacular? The false notion that the well-meaning idealist and self-serving demagogues who propound the free market aren't full of as much crap as can be, but they manage to dazzle the public with their own empty rhetoric.
Then they are people who understand insurance as poorly as you appear to. The healthy pay the unhealthy's medical bills because a) the risk and cost is shared by all premium payers when they are healthy (including those who get sick later) b) the unhealthy don't need the added stress of getting huge bills to impact their attempts to get better and usually can't work while they're in traction or a coma.
You know, sort of like how in fire or flood insurance the people whose houses are standing help pay for replace the houses that got burnt down/flooded/washed away (but whose owners had paid insurance premiums). That's how insurance works!
Laissez lire, et laissez danser; ces deux amusements ne feront jamais de mal au monde. - Voltaire
It happens occasionally here in the Southwest U.S. that European tourists forget they can drive for eleven hours and not see anyone, or anything. Including a petrol station.
I did not say, it does not get done at all — I said, it is done poorly. The government-managed highways and bridges suck — next time you are stuck in traffic, you'll be forced to agree with me....
In Soviet Washington the swamp drains you.
Is not it a little early in the conversation for ad hominems?
Citation needed?
A person of Ronald Reagan's station will get the very best care available in any country and under any regime.
Several considerations destroy this argument too:
Everybody, who works for government grants hates the process with passion. Various writers (including stars like Asimov) mocked the it viciously.
Are you honestly not aware of the numerous problems with this wonder — where everything is spoofable and nothing is encrypted? Where all sorts of data travels in clear text and security considerations are still — decades later — being bolted on?
Yes, they did. Poorly... Billions of dollars to take 3 men there and back — with a handful of rocks... Where are the lunar settlements? Where are the retirement homes for the elderly to enjoy the improved quality of life in lower gravity? The shuttle-program — after consuming the untold more billions of dollars — is scuttled, we are using Russian vehicles to bring stuff up. And the Russian are government made too — just cheaper, because everything is cheaper in a poor country.
Heinlein argued before a congressional committee in favor of funding space-travel and related research. But in his books it was the entrepreneurs — motivated by both profit and passion — who did the exploration. If those billions were allowed to stay in the private hands — rather than be taxed away — could there have been a Luna Hilton up there by now?
Military organizations, by the very nature of the domain, are not easily subject to competitive markets — government's monopoly must exist there. And yet... The security organizations like Black Water (currently known as Academi) have shown, how much better a privately organized force can be — for far less money — than an official military.
Government sucks at everything — an efficient government is a dictatorship said John Kennedy. Some things can not be done outside of government. But whatever can, should...
In Soviet Washington the swamp drains you.
Cool story, bro. I could've asked for some references, but it does not change a thing. The point was — and remains — that Amtrak sucks. The link I gave earlier discusses — as an example — the cost of a can of soda... Despite selling it to passengers for $2, Amtrak loses money, because their own cost is $3.40 per can... Nobody can explain this — not only can you buy the cans in the supermarket for $.30-.50, even privately-run vending machines on each station sell the same cans for less than $2. Their proprietors would've been ecstatic to supply the passing trains with as much as they could take... But no, for some reason, Amtrak's costs are $3.40 — would not you love to be their supplier? Somebody is...
Darling, whatever you are trying to say here is decidedly not part of vernacular...
In Soviet Washington the swamp drains you.
You — and others — seem to have misconstrued my argument to mean, the government simply can not do anything. That's not, what I said. They can do it — just poorly.
Is not it a little early in the conversation for ad hominems?
I base this on several years of the Wall Street Journal comments page, until I gave up on them. And I read lots of conservative articles on health care policy. There are conservatives who change their minds based on the facts, but in my experience they are rare. William Buckley is dead. The WSJ editorial page has turned into a Pravda for the conservative wing of the Republican Party. Sic transit gloria mundi.
The military and Veterans Affairs medical centers give some of the best care in the world
Citation needed?
http://www.ncbi.nlm.nih.gov/pubmed/?term=%22Veterans'+Affairs%22
http://www.nejm.org/doi/full/10.1056/NEJMoa1007474
http://www.ncbi.nlm.nih.gov/pubmed/7979780
http://circ.ahajournals.org/content/86/1/121.abstract
http://circ.ahajournals.org/content/93/12/2128.abstract
In case you're not used to reading medical journal articles (and most people aren't), the point of these studies is that they took the medical conditions that they most frequently treated, and were responsible for the most deaths, like heart disease, high blood pressure, and kidney disease, where different doctors treated the same patients different ways, and they did randomized, controlled trials to see which treatments worked at all and which were better. They also did studies of different VA hospitals to see which hospitals had better and worse outcomes. They tried to improve the hospitals with worse outcomes, and if that didn't work, they shut the departments down.
If you go to any major medical conference, and go to the sessions on important diseases, you'll usually hear them talking about the "VA study." That's because in many medical specialties, the VA did the major, best-designed study to find out which treatments work and didn't work. There are a few private non-government organizations, like Kaiser-Permanente and Blue Cross/Blue Shield, who do the same thing, but (not to disparage them), the VA does a lot more of these studies.
The National Institutes of Health also does big studies like that. Of course, with the budget cuts, they can't do as many, and they're being forced right now to decide which important ongoing studies will have to go, as Science and Nature have been reporting.
Everybody who follows medical research knows this. If you say, the government can't do anything well, they'll know that you don't know anything about the reality in this important field.
And as for those complaints about the bad outcomes in VA hospitals -- those are the kind of thing that happen in any hospital. It's easier to find out what happens in the VA hospitals because of their internal accounting and disclosure policies. You'll notice that the story got that information from the government's own review. Try to get that same information from private hospitals. What matters is when doctors who know how to compare hospitals compare large numbers of patients, to see whether there are any statistically significant patterns. When they do that, the VA hospitals usually do well. And when they don't, they find out why and how to change it.
https://en.m.wikipedia.org/wiki/Test-driven_development is better for Govt projects
Casteism
Given the amounts of money the government spends on this sort of research, it is not at all surprising, that few others do. Competing with government — like fighting the city hall — is usually fruitless and often dangerous. And not because the government does such a good work, but because their pockets are much deeper and, when that does not help, they can outlaw you.
VA may have terrific research, but we will never know, if anyone would've done better — given the same sort of money. As for their actual treatment of patients — rather than fundamental research — the cheating of veterans is as widespread as is often attributed to the insurance companies.
And, again, I posit that this could well be due to the government doing, what it should not be. Whether it is doing it poorly (as I suspect) or not, competing with the government is rarely practical and those geniuses choose to apply themselves elsewhere. Why are there geniuses — of Randian proportions — in the field of computers and other electronics, for example, but not in medicine?
No surprise there — large corporations (merk and krupps of the world) would usually rather cooperate with the government. For the rest of us the arrangement may be suboptimal.
Is not obvious by now, that the "big expensive" project of traveling to the Moon not only did not have an immediate return — it had no return at all? Are you still sure, you want to insist on it as evidence of the government doing useful things better than private entities?
The Soviet Union's ongoing failure (and I grew up there myself) is not due to any flaws in the free market system. It was due to their trying to use it as a tool only. You can't simply institute a free market — it has to grow out of personal freedoms (and respect for them), as in the US. That said, the freer parts of the former USSR (such as the Baltic republics) and the members of "Warsaw Block" are doing fairly well — because they spent 1-2 generations less under the Communist rule. Those dark decades will take a long time to undo...
But if we must refer to examples of other countries, I encourage to compare North and South Koreas, Soviet Estonia vs. Finland, East and West Germany. Identical people and culture. Government vs. free enterprise. See, who wins...
In Soviet Washington the swamp drains you.
What's "So Mo"? Southern Missouri? Technically that would be "So MO" to disambiguate from Montana...
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
Democrats wanted it, so it was doomed to fail since nothing they do work.
So all those bills the Republicans have been desperately voting down "BECAUSE DEMONCRATS!!" are just mercy killings?
Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
Why? Is Europe's location somehow significant to average Americans?
Well you know, you might like to bomb us sometime. Then it comes in handy to know where it is!
Stefan Axelsson