How Healthcare.gov Changed the Software Testing Conversation

An anonymous reader notes an article about how the tribulations of Healthcare.gov brought the idea of software testing into the public consciousness in a more detailed way than ever before. Quoting: "Suddenly, Americans are sitting at their kitchen tables – in suburbs, in cities, on farms – and talking about quality issues with a website. The average American was given nightly tutorials on load testing and performance bottlenecks when the site first launched, then crumbled moments later. We talked about whether the requirements were well-defined and the project schedule reasonably laid out. We talked about who owns the decision to launch and whether they were keeping appropriate track of milestones and iterations. ... When the media went from talking about the issues in the website to the process used to build the website was when things really got interesting. This is when software testers stepped out of the cube farm behind the coffee station and into the public limelight. Who were these people – and were they incompetent or mistreated? Did the project leaders not allocate enough time for testing? Did they allocate time for testing but not time to react to the testing outcome? Did the testers run inadequate tests? Were there not enough testers? Did they not speak up about the issues? If they did, were they not forceful enough?"

Comments

WTF Not forceful enough?

[Anonymous Coward]

Ya those damn testers, they just can't communicate the issues to management. Like that NASA engineer and the O-rings. Stop blaming the testers.

Re:WTF Not forceful enough?

[ObsessiveMathsFreak]

Well you either blame the help, or else someone important is going to have to take the blame.

development not complete

[BradMajors]

It is not really a question of testing. Parts of the software were missing or incomplete. You can't test what isn't there.

Re:development not complete

[ranton]

The real world does not work like this and I find the best work comes when the guy who will be writing the software actually collects the requirements and does a good job of that including a plan of how to test the code.

The worst code I see is written when the programmer is given some narrow requirements on some ticket and they code directly to those requirements with little to no knowledge of the overall system they are working on. In some magical world where all of your requirements are perfect this could work very well. But part of being a solid developer is knowing how to spot the "smell" of bad requirements. While sometimes you can do this without any knowledge of the overall system, you are much more likely to have the right insights if you have some relevant domain knowledge.

I have a current project that I am maintaining where I wrote most of the code in a vacuum without understanding the customer's real needs. It is a horrible mess and I would have done things completely differently if I was involved in meetings with the clients early on. I'm not saying my code would have been perfect, but there were some massive disconnects between the assumptions I drew from the written requirements and the explanations I got from the clients once I was given more access to them. Now there is no time or money for massive rewrites or refactorings, so it continues to be a thorn in my side.

Re:Were known management tools used?

[Anonymous Coward]

How do you make a presentation of working software when the working software is designed to pull data from half a dozen databases outside of your control? You might as well throw a football in the air and then claim you're a world class quarterback, you're just waiting for the rest of the world class team to show up and assist you.

Someone important *should* take the blame

[Anonymous Brave Guy]

One of the most insightful truths ever told to me:

It is always management's fault.

This goes right to the root of the tree, because by definition if the people further out couldn't get the job done or didn't have the right resources to do it, it was management's responsibility to fix those problems. The buck stops with the most senior managers on a project, whose only two choices are to explain what is needed to succeed and then do so if given those things, or to fail.

Re:"Average American was given nightly tutorials o

[blackpaw]

Why? Is Europe's location somehow significant to average Americans?

And there is the proof of the OP's implied statement

Re:"Average American was given nightly tutorials o

[Oligonicella]

A Swiss friend of mine visited DC. I live in So Mo. We chatted on the phone and he suggested we get together for lunch the next day. So, I agreed and told him the city in Virgina we could meet in after driving eleven hours.

They're no better with our geo than we are with theirs.

Funny thing: testing is not that important...

[gweihir]

If you find out in testing that your architecture or design does not cut it, you are screwed. The only thing you can usually do is scrap the project and start again. Testing does only work for simple things like simple busiess logic and the like, where you know the characteristics very well beforehand. For anything that is a new design, the only thing that helps is very capable and experienced architects and designers that have a good change to get it right by intuition. This will be people that can do architecture, design and implementation and can do all three well. Not many of those exist, but there is no replacement for them. Those that think they can do things on the cheap without not only having this type of expert but also listening to them closely will fail. This can be observerd time and again and can alost be called a "well established industrial practice", because quite a few "managers" do not actually know that it can be done better. Funny thing, in other fields, you have chief enineers, architects and the like and the critical work is not given to people that are likely to fail. Only IT messess it up regularly, because talent and exerence is not respected.