Slashdot Mirror
Should Developers Do All Their Own QA? (itnews.com.au)
An anonymous reader quotes IT News:
Fashion retailer The Iconic is no longer running quality assurance as a separate function within its software development process, having shifted QA responsibilities directly onto developers... "We decided: we've got all these [developers] who are [coding] every day, and they're testing their own work -- we don't need a second layer of advice on it," head of development Oliver Brennan told the New Relic FutureStack conference in Sydney last week. "It just makes people lazy..."
Such a move has the obvious potential to create problems should a developer drop the ball; to make sure the impact of any unforeseen issues is minimised for customers, The Iconic introduced feature toggles -- allowing developers to turn off troublesome functionality without having to deploy new code. Every new feature that goes into production must now sit behind one of these toggles, which dictates whether a user is served the new or old version of the feature in question. The error rates between the new and old versions are then monitored for any discrepancies... While Brennan is no fan of "people breaking things", he argues moving fast is more beneficial for customers.
"If our site is down now, people will generally come back later," Brennan adds, and the company has now moved all of its QA workers into engineering roles.
Such a move has the obvious potential to create problems should a developer drop the ball; to make sure the impact of any unforeseen issues is minimised for customers, The Iconic introduced feature toggles -- allowing developers to turn off troublesome functionality without having to deploy new code. Every new feature that goes into production must now sit behind one of these toggles, which dictates whether a user is served the new or old version of the feature in question. The error rates between the new and old versions are then monitored for any discrepancies... While Brennan is no fan of "people breaking things", he argues moving fast is more beneficial for customers.
"If our site is down now, people will generally come back later," Brennan adds, and the company has now moved all of its QA workers into engineering roles.
We're lazy as shit.
"It just makes people lazy..."
Totally agree. I have lobbied for developers to build integration tests against their code. I started by lobbying for QA to do this but they weren't technical enough and couldn't handle it. The run-of-the-mill QA process is a fluffy user acceptance test. I have proof. We continue to find the real problems in production.
I object to power without constructive purpose. --Spock
QA done right can add value to a project by rooting our ambiguous requirements and holding developers accountable. Developers are worried about a lot of issues like function, aesthetics, security, performance, scalability. Having QA work from the same specs to derive the functional tests from a different perspective can provide a huge value to developers. But you have to involve QA early in the game.
No, for the same reason professional writers need an editor. Someone else will see things that you didn't. Yes, they will.
Didn't Microsoft move to this model just before the Windows 10 upgrade, uh, issues?
https://www.windowscentral.com...
I've done development and QA work, and QA approaches testing from from a whole different perspective. The problem is context switching. It's difficult for a developer to approach a piece of functionality from the user's perspective (especially if they don't necessarily know all the parameters for how the functionality is going to be used) It takes time to switch out of that mode, and, in an effort to make the code perfect, you can come short of ever making the code good to begin with.
A QA engineer is going into testing with the mindset of an end user. What are they going to expect to have happen? What are they likely to do? What corner cases aren't worth exploring? It's useful for developers to have some experience in this, but I don't think it's reasonable to expect them to be experts.
In general, you want developers to spend time and cognitive effort making sure the code is functional and maintainable. Heaping on the cognitive load of completely switching gears and thinking like an end user isn't always the best use of their resources.
My Other Computer Is A Data General Nova III.
Longer version: NOOOOOOOOOOOOOOOOOOO.
I've been at places where managers get a variation on this lovely idea: Hey, maybe you would be more careful coders if you had to do your own testing. You're all so sloppy!
Oddly enough, this is mostly from non-coders, or folks without technical acumen.
Here's the deal: There ARE coders that claim to not make mistakes, and create bug-free code. Those are known as inexperienced or self-deluded coders, or folks only making one-off projects negotiating exact terms that make it not REALLY programming so much as configuring.
Even just from a simple level as just writing down words, if you've ever written a work longer than a couple of paragraphs, you'll virtually always reach a point where there's some subtle or stupid mistake you make without noticing it. Even VERY experienced journalists and writers need proofreaders on any serious project.
It's not a matter of taking your work seriously, or being unable to face responsibility - you're always going to be blind to the flaws in your own code, and although testing crews ARE expensive - they exist for many, MANY important reasons. They're cheaper than the same churn without a proper quality process. Sure, some of that effort may be being duplicated, or used badly - but few possibilities would be worse than expecting everyone to 'man up' and expect bug count to go down.
In other words - certainly, I can test my own code if you'd like, but I'm a VERY expensive testing staff member, and my perspective is going to cause me to overlook a lot of things - since it already 'makes sense' to me and I won't spot the inconsistencies a percent of the time.
Ryan Fenton
EVERYONE on your team should be a developer.
But QA should come from that pool for one project. Swap them out for the next, etc.
Developers working at QA know where the mistakes are likely to be.
Developers testing their own shit subconsciously avoid the shoddy parts.
This also keeps the team as one.
It's like a platoon of Marines taking turns burning the shit.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
This is dumb.
QA people think like QA people. Developers think like developers. And developers can't bring a fresh eye to their own code.
Heck, most developers can't even develop remotely comprehensive test data.
There is some evidence in the nuclear and aerospace industries that having things checked twice results in people being lazier (because they think the other person will catch it).
But for mainstream business developers, I've never seen them able to test as well as QA does.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
I can say, as a lead developer and architect at several companies over the last 12 years, having a good QA team to catch all the edge cases makes me a much more productive developer. I have worked in many situations where I didn't have Quality Assurance, and the quality for when I did have QA was much much better.
External testing is necessary to avoid a number of cognitive biases. Developers should do some degree of testing new builds, especially if the tests can be automated, but at the very least, you at least need roughly the software equivalent of a proofreader. Proofreaders are needed, not because a writer lacks the ability to understand basic grammar and spelling rules, but because the brain suffers from the opposite of NIH syndrome.
Does that necessarily have to take the form of a QA department? No, but this mostly sounds like trying to cut manpower or get around addressing flaws in the testing process. Another PHB trying to preserve his existence by pulling out pieces of a machine they don't understand.
This is my signature. There are many like it, but this one is mine.
I've done QA and I moved to development. The jobs are very different. Devs should check their own work and write unit tests, obviously, but QA really is it's own step and should not be minimized. Sure, you can code regression testing and unit test just about anything in the right environment, but a good QA tester will do things you will never think of and catch a bug before it is found on production.
I code in 3 different languages and write my own QA in each, but only because THERE IS NO QA department. Writing useful QA always takes way longer than initial coding of the script to do its intended purpose.
Coder's lament: "Why would a user *ever* do that?"
Because they are a different set of eyes and don't do the same things the coder does.
2 weeks ago I torpedoed my own QA handling an important script that had been working fine, QA 100% successful (so my report said). A "simple", backwardly compatible enhancement entailed a QA change in order to QA the new feature. For a whole day I was ecstatic that the QA **still** passed 100%.
Next day .. that sleep I got .. brought me to my senses .. I'm just not THAT good of a coder going back to an older script. A skeptical look revealed that the QA's "simple" change was allowing EVERYTHING to pass!
QA is /not/ good if it's done by the same person. (And you don't want to wait 6 weeks so that YOU are /not/ the same person you were, and THEN write the QA. Nice try..)
Everywhere it seems: Gotta have this done last week; call it what you want -- *Lie* to the customer.. they are too naive to complain.. too wet under the ears to blame YOU instead of themselves.)
Insert your own diatribe vvvvv here vvvvv.
Yeech.
That was Zen, this is Tao
QA is needed. However far too often QA teams are used as dumping grounds for Developers who cannot cut it
In my perspective the most complicated part in actually figuring out the tests. if you can figure out the tests it means you have nailed down the scope. if you can do that before you start coding the coding part is actually easy. However if you are only doing QA but not doing coding your coding skills will rust.
The best place I have worked had 2 scrums working on a project - Team Blue and Team Orange. For one release Team Orange was dev and Team Blue was QA. For next release Team Blue was Dev and Team Orange was QA. They kept switching around keeping both their Dev and QA skills updated but at the same time the context switch is easier to do at 6 weeks intervals than continuoisly during the day.
**Life is too short to be serious**
"Test is dead" was the keynote presentation of a Google Test Automation Conference six years ago.
My personal view is that if you are doing web development where your company rapidly & repeatedly deploys releases on behalf of customers, you might be able to get away with not having much of a QA department so long as the impact is low. If a problem arises it can be quickly fixed without much of a financial loss.
But if you are in a regulated industry, failure of your software will result in significant lost revenue while its being fixed, or your software is deployed only every few weeks or months by customers who do their own acceptance and integration testing, then you probably need to do more QA work & dedicated QA work upfront. In such scenarios the software producer and their customers may encounter significant losses and/or inconvenience because something faulty snuck through.
Developers should be required to QA all their own code, but there still needs to be a second QA pass done by someone who did not write it. Basic human psychology causes natural blind spots in one's own logical thought processes. It's impossible to be completely unbiased about stuff you wrote yourself. So, while a developer is the best person to QA their own work, they're also usually inherently incapable of completing the task with reasonable accuracy in a reasonable time frame. Therefore, 2 people at minimum should evaluate every bit of code, but yes, one of them should be the primary maintainer of the code and the other one should be someone from a different team.
This has to be the absolute worst idea I've heard in a long time. Decent developers generally don't make good testers. Developers don't test their own code well -- I suspect because they know the workflow and don't try enough out of the box things. I'm not suggesting that the developers should be absolved from testing, they should write unit tests, end to end tests and everything in the middle. I am suggesting that a second tier of testers is a huge improvement.
A corollary, if you ever have a good or great test person on staff, keep them. Sure, they'll piss off some of the developers but they are worth their weight in saffron.
Not for their own code. Developers are lousy at testing their own code, because they handle the errors that they think of, and won't make the ones they don't.
I suspect that developers are even poor choices for testing other developers code, because they're less likely to make ID10T errors.
I think we've pushed this "anyone can grow up to be president" thing too far.
I've been a software engineer, a QA engineer, traditional IT SysAdmin, and have now found my niche in SysOps/DevOps/SRE/whatever-we're-calling-it-this-week. And good lords of Kobol, QA is absolutely necessary. The usual reasons definitely apply. A separate pair of eyes will find things you've been blind to because you're too close to your own work. Building things and breaking things are different skillsets and different mindsets. There will be edge cases that even the spec didn't anticipate. Blah blah blah.
But try being on the operations front line some time, and you'll REALLY appreciate a good QA team. Right now, I'm responsible for multiple products from teams that, due to acquisitions and re-orgs and different companies doing things differently, some have dedicated QA and some let the developers test their own code. I see the errors and failures before anyone else. And the QA'd stuff invariably causes me less grief than the "we trust the developers" stuff. Hell, if I had my way, we'd have QA vet ops changes and updates too; because I'm under no illusions of infallibility myself.
Imagine all the people...
As a developer solo-managing a legacy codebase, it is still important to have various rounds of external QA.
The first issue is that the developer can be so far into the code that they can completely miss what is obvious or un-intuitive to a non-technical user. This may also involve legacy bugs or interactions with code they havn't written themselves.
The second issue is that the the spec itself may not have been fully defined, or what you have is the developer's interpretation of the spec, and this may require clarification or feedback once someone has seen the end result.
The third issue is that a developer will focus their attention on the things they consider most important. Sometimes the best way to achieve this is to just give a developer a list of minor bugs/features, which is a great way of focusing attention by clarifying the spec and letting a developer speed though a bunch of quick fixes which have a predefined spec (defining the spec is half the mental effort).
So in general a developer should be capable of doing a first round of internal QA on their own code, but a second pair of eyes is still occasionally needed, as the developer has a very peculiar set of perception filters.
Developers should absolutely have unit and integration tests and only deliver a product to QA that they think will be immediately passed onto the customer. But within many organizations this just doesn't make people happy, and unhappy people make bad decisions. QA can give a seal of approval that gets better buy in within the company. This should not be the way it is but that's life.
Lastly, a good QA will look to see that what was produced is what was actually asked for and promised. This is no small thing, because many programming teams can lose sight of what the contract actually says. "But it's better" is something that should be negotiated, not just delivered.
But that said, QA can be a major road block. It is not uncommon that you throw things over the fence only to find that QA has hyper focused on something that isn't a requirement. For instance you might have some system that does echo location to find the distance to a wall. Your requirements call for one successful echo location every 10 seconds. You fire out 10 per second and around 50% fail. The reality is that you are getting roughly 5 ranges per second and never will you miss your target of one every 10 seconds. Yet QA gets all wound up about the 50% failure rate and reports that the product has a 50% failure rate. This is where you begin convincing the accountants that the QA department needs a head count reduction.
In my experience, QA may get paid less, but they work harder than the developers.
In my experience, Bricklayers may get paid less, but they work harder than the QA.
IOW, you don't have a point. We don't pay people more for working harder. We pay them for the value they bring.
The more high-quality devs you have, the lower the value of the QA as there's fewer issues to be found and reported.
A place with shitty copy-paste devs needs better QA to stop unsuitable product from leaving the company, so the QA there is more valuable.
I'm a minority race. Save your vitriol for white people.
It's cheaper. Did we mention that? It's 'way cheaper.
I've calculated my velocity with such exquisite precision that I have no idea where I am.
In aerospace, independent verification and validation is a critical part of the regulatory process. Every change has a reason, every reason is documented, and every change is reviewed by someone else.
[
... and not from a company that makes stuff that kills people when it malfunctions. Airplanes, autonomous vehicles, medical devices, etc.
Should Developers Do All Their Own QA?
If you can do that without deleterious consequences, then your work is trivial and/or not overtly complex.
Otherwise, such a thing is insane, specially if you work in developing actual products (shrink-wrapped software or software bundled in hardware), or complex services. ) And let us not even mention software related to avionics, weapons or medical devices (where QA is done by a separate organization to remove bias.)
The primary concern is always bias, bias that come from first-hand knowledge of a thing (or assumptions about a thing.)
It is a lot easier and effective to have a 3rd party dedicated to methodical black box and end-to-end integration/UA testing than to get a developer to test his own creation to the breaking point.
There is a reason why organization hire external teams to perform penetration testing of their infrastructure. He who knows nothing of the internals a-priori is the perfect person to black-box test the shit out of something with nothing but specs.
This is really a lame-brain position. As a long time developer I have two mottos:
* If it isn't tested, it doesn't work.
* A developer make assumptions all the time.
I can guarantee that, if I was responsible for QA on my own code, things would be broken in subtle ways.
Many years ago I did QA consulting for AT&T. We set up a system where the human factors engineering group would document an application in a specification. One copy went to the developers and one copy went to the QA personnel. The QA staff designed a test plan from this spec and then implemented it against the product sent from the developers. I can't tell you how many things the developers missed because of assumptions they made that were caught by the test plans.
Peer review is great. I introduced it at my company. It's not QA.
Users do stupid shit, shit no programmer in their right mind would ever think of doing. As an extremely simplistic example, at my job we have a script that calls nap with various arguments, including a list of IPs provided by user. The programmer tested it with some IPs. The peer could test it with some other IPs. The programmers made sure it works. Users will pass it 0.0.0.0/0, and fa70:638:363:7363::76. QA does the same, because QA is trying to break it. Programmers ensure it works, QA figures out how it can be broken (and are therefore an important part of security).
Aside from that difference, excellent programmers (10% of programmers) will PRODUCE quality. QA will ASSURE quality. With programmers testing, you can hope the quality is good. With QA testing, you can know the quality is good.
When does Bob start writing horrible code, knowing that Margareth will find the bugs and clean it up for him? And when does Bob get the promotion because his code shipped faster, and Margareth's slower developed code fails to be presented early? So Bob gets to go to luch with the boss, go tot the company picnics, and gets invited to design review meetings because "they ship faster"?
I've seen this happen. It's been done to me, as "Margareth" in the story. I eventually had to talk with my manager, from quite a long time ago, and warn that manager what was going to happen to Bob's workflow and code quality when I went on vacation.