Slashdot Mirror
Should Developers Do All Their Own QA? (itnews.com.au)
An anonymous reader quotes IT News:
Fashion retailer The Iconic is no longer running quality assurance as a separate function within its software development process, having shifted QA responsibilities directly onto developers... "We decided: we've got all these [developers] who are [coding] every day, and they're testing their own work -- we don't need a second layer of advice on it," head of development Oliver Brennan told the New Relic FutureStack conference in Sydney last week. "It just makes people lazy..."
Such a move has the obvious potential to create problems should a developer drop the ball; to make sure the impact of any unforeseen issues is minimised for customers, The Iconic introduced feature toggles -- allowing developers to turn off troublesome functionality without having to deploy new code. Every new feature that goes into production must now sit behind one of these toggles, which dictates whether a user is served the new or old version of the feature in question. The error rates between the new and old versions are then monitored for any discrepancies... While Brennan is no fan of "people breaking things", he argues moving fast is more beneficial for customers.
"If our site is down now, people will generally come back later," Brennan adds, and the company has now moved all of its QA workers into engineering roles.
Such a move has the obvious potential to create problems should a developer drop the ball; to make sure the impact of any unforeseen issues is minimised for customers, The Iconic introduced feature toggles -- allowing developers to turn off troublesome functionality without having to deploy new code. Every new feature that goes into production must now sit behind one of these toggles, which dictates whether a user is served the new or old version of the feature in question. The error rates between the new and old versions are then monitored for any discrepancies... While Brennan is no fan of "people breaking things", he argues moving fast is more beneficial for customers.
"If our site is down now, people will generally come back later," Brennan adds, and the company has now moved all of its QA workers into engineering roles.
We're lazy as shit.
"It just makes people lazy..."
Totally agree. I have lobbied for developers to build integration tests against their code. I started by lobbying for QA to do this but they weren't technical enough and couldn't handle it. The run-of-the-mill QA process is a fluffy user acceptance test. I have proof. We continue to find the real problems in production.
I object to power without constructive purpose. --Spock
QA done right can add value to a project by rooting our ambiguous requirements and holding developers accountable. Developers are worried about a lot of issues like function, aesthetics, security, performance, scalability. Having QA work from the same specs to derive the functional tests from a different perspective can provide a huge value to developers. But you have to involve QA early in the game.
No, for the same reason professional writers need an editor. Someone else will see things that you didn't. Yes, they will.
Didn't Microsoft move to this model just before the Windows 10 upgrade, uh, issues?
https://www.windowscentral.com...
I've done development and QA work, and QA approaches testing from from a whole different perspective. The problem is context switching. It's difficult for a developer to approach a piece of functionality from the user's perspective (especially if they don't necessarily know all the parameters for how the functionality is going to be used) It takes time to switch out of that mode, and, in an effort to make the code perfect, you can come short of ever making the code good to begin with.
A QA engineer is going into testing with the mindset of an end user. What are they going to expect to have happen? What are they likely to do? What corner cases aren't worth exploring? It's useful for developers to have some experience in this, but I don't think it's reasonable to expect them to be experts.
In general, you want developers to spend time and cognitive effort making sure the code is functional and maintainable. Heaping on the cognitive load of completely switching gears and thinking like an end user isn't always the best use of their resources.
My Other Computer Is A Data General Nova III.
Longer version: NOOOOOOOOOOOOOOOOOOO.
I've been at places where managers get a variation on this lovely idea: Hey, maybe you would be more careful coders if you had to do your own testing. You're all so sloppy!
Oddly enough, this is mostly from non-coders, or folks without technical acumen.
Here's the deal: There ARE coders that claim to not make mistakes, and create bug-free code. Those are known as inexperienced or self-deluded coders, or folks only making one-off projects negotiating exact terms that make it not REALLY programming so much as configuring.
Even just from a simple level as just writing down words, if you've ever written a work longer than a couple of paragraphs, you'll virtually always reach a point where there's some subtle or stupid mistake you make without noticing it. Even VERY experienced journalists and writers need proofreaders on any serious project.
It's not a matter of taking your work seriously, or being unable to face responsibility - you're always going to be blind to the flaws in your own code, and although testing crews ARE expensive - they exist for many, MANY important reasons. They're cheaper than the same churn without a proper quality process. Sure, some of that effort may be being duplicated, or used badly - but few possibilities would be worse than expecting everyone to 'man up' and expect bug count to go down.
In other words - certainly, I can test my own code if you'd like, but I'm a VERY expensive testing staff member, and my perspective is going to cause me to overlook a lot of things - since it already 'makes sense' to me and I won't spot the inconsistencies a percent of the time.
Ryan Fenton
EVERYONE on your team should be a developer.
But QA should come from that pool for one project. Swap them out for the next, etc.
Developers working at QA know where the mistakes are likely to be.
Developers testing their own shit subconsciously avoid the shoddy parts.
This also keeps the team as one.
It's like a platoon of Marines taking turns burning the shit.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
This is dumb.
QA people think like QA people. Developers think like developers. And developers can't bring a fresh eye to their own code.
Heck, most developers can't even develop remotely comprehensive test data.
There is some evidence in the nuclear and aerospace industries that having things checked twice results in people being lazier (because they think the other person will catch it).
But for mainstream business developers, I've never seen them able to test as well as QA does.
She was like chocolate when she drank... semi-sweet at first and then increasingly bitter.
I can say, as a lead developer and architect at several companies over the last 12 years, having a good QA team to catch all the edge cases makes me a much more productive developer. I have worked in many situations where I didn't have Quality Assurance, and the quality for when I did have QA was much much better.
External testing is necessary to avoid a number of cognitive biases. Developers should do some degree of testing new builds, especially if the tests can be automated, but at the very least, you at least need roughly the software equivalent of a proofreader. Proofreaders are needed, not because a writer lacks the ability to understand basic grammar and spelling rules, but because the brain suffers from the opposite of NIH syndrome.
Does that necessarily have to take the form of a QA department? No, but this mostly sounds like trying to cut manpower or get around addressing flaws in the testing process. Another PHB trying to preserve his existence by pulling out pieces of a machine they don't understand.
This is my signature. There are many like it, but this one is mine.
I've done QA and I moved to development. The jobs are very different. Devs should check their own work and write unit tests, obviously, but QA really is it's own step and should not be minimized. Sure, you can code regression testing and unit test just about anything in the right environment, but a good QA tester will do things you will never think of and catch a bug before it is found on production.
I code in 3 different languages and write my own QA in each, but only because THERE IS NO QA department. Writing useful QA always takes way longer than initial coding of the script to do its intended purpose.
Coder's lament: "Why would a user *ever* do that?"
Because they are a different set of eyes and don't do the same things the coder does.
2 weeks ago I torpedoed my own QA handling an important script that had been working fine, QA 100% successful (so my report said). A "simple", backwardly compatible enhancement entailed a QA change in order to QA the new feature. For a whole day I was ecstatic that the QA **still** passed 100%.
Next day .. that sleep I got .. brought me to my senses .. I'm just not THAT good of a coder going back to an older script. A skeptical look revealed that the QA's "simple" change was allowing EVERYTHING to pass!
QA is /not/ good if it's done by the same person. (And you don't want to wait 6 weeks so that YOU are /not/ the same person you were, and THEN write the QA. Nice try..)
Everywhere it seems: Gotta have this done last week; call it what you want -- *Lie* to the customer.. they are too naive to complain.. too wet under the ears to blame YOU instead of themselves.)
Insert your own diatribe vvvvv here vvvvv.
Yeech.
That was Zen, this is Tao
QA is needed. However far too often QA teams are used as dumping grounds for Developers who cannot cut it
In my perspective the most complicated part in actually figuring out the tests. if you can figure out the tests it means you have nailed down the scope. if you can do that before you start coding the coding part is actually easy. However if you are only doing QA but not doing coding your coding skills will rust.
The best place I have worked had 2 scrums working on a project - Team Blue and Team Orange. For one release Team Orange was dev and Team Blue was QA. For next release Team Blue was Dev and Team Orange was QA. They kept switching around keeping both their Dev and QA skills updated but at the same time the context switch is easier to do at 6 weeks intervals than continuoisly during the day.
**Life is too short to be serious**
"Test is dead" was the keynote presentation of a Google Test Automation Conference six years ago.
My personal view is that if you are doing web development where your company rapidly & repeatedly deploys releases on behalf of customers, you might be able to get away with not having much of a QA department so long as the impact is low. If a problem arises it can be quickly fixed without much of a financial loss.
But if you are in a regulated industry, failure of your software will result in significant lost revenue while its being fixed, or your software is deployed only every few weeks or months by customers who do their own acceptance and integration testing, then you probably need to do more QA work & dedicated QA work upfront. In such scenarios the software producer and their customers may encounter significant losses and/or inconvenience because something faulty snuck through.
I suppose if you donâ(TM)t care if your site and backend stop working one day then having devs do QA is fine. It all depends on your requirements and expectations. Oh, and who checks to see that toggle a doesnâ(TM)t affect code path c? End-users, of course.
Why? All my code compiles and runs perfectly the first time.
But seriously, the first level of QA should be the developer. Before it leaves the developer, the code should work reasonably well or at leave pass some sanity checks. Not, "it compiles and links, so I'm done."
Fight Spammers!
This is the problem I see: Developers are inherently held to a higher standard than everyone in a dev shop. Their code must pass the parser, the compiler, and execute properly. This means not a single typo. No errant semicolons or brackets. Every character must me right. Nobody else has that harsh or a standard. I've seen high level PR and corporate releases that have typos in the first sentence. Cringe-worthy stuff. Even the New York Times and Washington Post issue articles with typos in the headlines, daily. The world has become so sloppy, it's unforgivable. But developers can't get away with this, or the compiler craps out. So by inference, it's assumed that their flawless, meticulous attention to detail is universal, and not subject to question. Hence it falls further and further upon them to do their own internal code reviews, correction of flawed requirements, QA, documentation, and everything else around the code they write. Of course, because everyone else is too lazy and sloppy to produce the same level of results. I'm in a dev shop now where the dev coders are on the hook for everything from requirements to post-deployment support, and everything in between. It just happened because they're they only people with any attention to detail, and pride in what they do, and any thought beyond putting in 8 hours a day of work and collecting a paycheck. Welcome to the brave new world.
Developers should be required to QA all their own code, but there still needs to be a second QA pass done by someone who did not write it. Basic human psychology causes natural blind spots in one's own logical thought processes. It's impossible to be completely unbiased about stuff you wrote yourself. So, while a developer is the best person to QA their own work, they're also usually inherently incapable of completing the task with reasonable accuracy in a reasonable time frame. Therefore, 2 people at minimum should evaluate every bit of code, but yes, one of them should be the primary maintainer of the code and the other one should be someone from a different team.
The most productive team I've ever worked in had a simple policy.
1. You wrote your code.
2. You documented the developer's to your left's code.
3. You tested the developer to your right's code.
Harsh, but it worked really well.
But the developer to my left is an analog circuit designer and the developer to my right is a design automation expert. I do cryptography in the middle. I don't see this working out.
I should use this sig to advertise my book ISBN-13 : 978-1501515132.
Where I work, developers get paid a lot more than QA, so it would be silly to use developers as QA. Also, it is a different skillset. A good car mechanic isn't always a good driver.
If you run a shop without continuous integration/deployment, full testing automation, A/B deployment, constant feedback loops, etc... then you had better have a second pair of eyes (without the bias of a software developer) to purely represent the customer stakeholder. With those functions mentioned being in place however, it is entirely possible that the role of QA shifts strictly to that of test automation and monitoring of A/B deployment results. The majority of development houses only dream of such organizational maturity/sophistication - pie in the sky type stuff. By all means, keep reaching, but don't shoot yourself in the foot along the way...
This has to be the absolute worst idea I've heard in a long time. Decent developers generally don't make good testers. Developers don't test their own code well -- I suspect because they know the workflow and don't try enough out of the box things. I'm not suggesting that the developers should be absolved from testing, they should write unit tests, end to end tests and everything in the middle. I am suggesting that a second tier of testers is a huge improvement.
A corollary, if you ever have a good or great test person on staff, keep them. Sure, they'll piss off some of the developers but they are worth their weight in saffron.
Best results so far, embed the QA tester in the dev team.
Not for their own code. Developers are lousy at testing their own code, because they handle the errors that they think of, and won't make the ones they don't.
I suspect that developers are even poor choices for testing other developers code, because they're less likely to make ID10T errors.
I think we've pushed this "anyone can grow up to be president" thing too far.
Also known as "why do we need testers if we already have customers?"
Any guest worker system is indistinguishable from indentured servitude.
edge cases, schmedge cases. Not doing integration testing when everyone bleeds for their code and is personally invested in thinking that their code can't have a mistake because it's been tested. Oh, that's where the real fun begins.
Any guest worker system is indistinguishable from indentured servitude.
"If our site is down now, people will generally come back later," Brennan adds
In other words: we've conditioned our users to accept being shat upon.
A good QA person is going to use the system in ways you never thought of, thus never coded for. Your code might handle some of those un-thought of scenarios, and that is nice, but sometimes they will not...
If nothing else QA is the last line of defense also against shipping something that does not make sense to how users work. In development it's too easy to get into tasing a few small paths because the are mostly representative of all the possible ways someone may use your software, but in reality for the way real customers work, your workflow may be terrible. A QA person will know this (or should know this) and may have access to real production data they can run through.
I don't think QA has to be a huge role for a team, but I absolutely think there needs to be some QA person whose only job is QA, maybe a few light bug fixes and should probably understand code - but again it should be someone who is not a developer and who can think QA all the time.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
In my experience, QA may get paid less, but they work harder than the developers.
I've been a software engineer, a QA engineer, traditional IT SysAdmin, and have now found my niche in SysOps/DevOps/SRE/whatever-we're-calling-it-this-week. And good lords of Kobol, QA is absolutely necessary. The usual reasons definitely apply. A separate pair of eyes will find things you've been blind to because you're too close to your own work. Building things and breaking things are different skillsets and different mindsets. There will be edge cases that even the spec didn't anticipate. Blah blah blah.
But try being on the operations front line some time, and you'll REALLY appreciate a good QA team. Right now, I'm responsible for multiple products from teams that, due to acquisitions and re-orgs and different companies doing things differently, some have dedicated QA and some let the developers test their own code. I see the errors and failures before anyone else. And the QA'd stuff invariably causes me less grief than the "we trust the developers" stuff. Hell, if I had my way, we'd have QA vet ops changes and updates too; because I'm under no illusions of infallibility myself.
Imagine all the people...
As a developer solo-managing a legacy codebase, it is still important to have various rounds of external QA.
The first issue is that the developer can be so far into the code that they can completely miss what is obvious or un-intuitive to a non-technical user. This may also involve legacy bugs or interactions with code they havn't written themselves.
The second issue is that the the spec itself may not have been fully defined, or what you have is the developer's interpretation of the spec, and this may require clarification or feedback once someone has seen the end result.
The third issue is that a developer will focus their attention on the things they consider most important. Sometimes the best way to achieve this is to just give a developer a list of minor bugs/features, which is a great way of focusing attention by clarifying the spec and letting a developer speed though a bunch of quick fixes which have a predefined spec (defining the spec is half the mental effort).
So in general a developer should be capable of doing a first round of internal QA on their own code, but a second pair of eyes is still occasionally needed, as the developer has a very peculiar set of perception filters.
Developers should absolutely have unit and integration tests and only deliver a product to QA that they think will be immediately passed onto the customer. But within many organizations this just doesn't make people happy, and unhappy people make bad decisions. QA can give a seal of approval that gets better buy in within the company. This should not be the way it is but that's life.
Lastly, a good QA will look to see that what was produced is what was actually asked for and promised. This is no small thing, because many programming teams can lose sight of what the contract actually says. "But it's better" is something that should be negotiated, not just delivered.
But that said, QA can be a major road block. It is not uncommon that you throw things over the fence only to find that QA has hyper focused on something that isn't a requirement. For instance you might have some system that does echo location to find the distance to a wall. Your requirements call for one successful echo location every 10 seconds. You fire out 10 per second and around 50% fail. The reality is that you are getting roughly 5 ranges per second and never will you miss your target of one every 10 seconds. Yet QA gets all wound up about the 50% failure rate and reports that the product has a 50% failure rate. This is where you begin convincing the accountants that the QA department needs a head count reduction.
In my experience, QA may get paid less, but they work harder than the developers.
In my experience, Bricklayers may get paid less, but they work harder than the QA.
IOW, you don't have a point. We don't pay people more for working harder. We pay them for the value they bring.
The more high-quality devs you have, the lower the value of the QA as there's fewer issues to be found and reported.
A place with shitty copy-paste devs needs better QA to stop unsuitable product from leaving the company, so the QA there is more valuable.
I'm a minority race. Save your vitriol for white people.
It's cheaper. Did we mention that? It's 'way cheaper.
I've calculated my velocity with such exquisite precision that I have no idea where I am.
In aerospace, independent verification and validation is a critical part of the regulatory process. Every change has a reason, every reason is documented, and every change is reviewed by someone else.
[
The problem with this is that the programmer knows too much about the code. If there is a form involved, the developer knows what goes where, and if the field name is nonsense, he will still enter the right thing. If the field says 'Name', and the average joe enters his name here, and the developer wants the guys Userid... The developer won't catch this... the Q/A guy will enter hist name, not his userid, and the test will fail. There are tons of examples where the look and feel is wrong, or the error checking is nonsense, and the developer will often fail to catch these, because ... well it he thought of it, he would have fixed it or checked for it!
Q/A needs to be done by people that are not familiar with the product... This is a problem for even normal Q/A departments, because the guy doing the Q/A is probably the guy that did it for the last version, and now HE is too familiar with it too!
As for the customer doing the Q/A this is what betas are for. Real end use Q/A will always be done by customers no matter how much we wish it wasn't because of the problems above.
... and not from a company that makes stuff that kills people when it malfunctions. Airplanes, autonomous vehicles, medical devices, etc.
Should Developers Do All Their Own QA?
If you can do that without deleterious consequences, then your work is trivial and/or not overtly complex.
Otherwise, such a thing is insane, specially if you work in developing actual products (shrink-wrapped software or software bundled in hardware), or complex services. ) And let us not even mention software related to avionics, weapons or medical devices (where QA is done by a separate organization to remove bias.)
The primary concern is always bias, bias that come from first-hand knowledge of a thing (or assumptions about a thing.)
It is a lot easier and effective to have a 3rd party dedicated to methodical black box and end-to-end integration/UA testing than to get a developer to test his own creation to the breaking point.
There is a reason why organization hire external teams to perform penetration testing of their infrastructure. He who knows nothing of the internals a-priori is the perfect person to black-box test the shit out of something with nothing but specs.
But for mainstream business developers, I've never seen them able to test as well as QA does.
I've been developing complex software systems for about 30 years now, and I couldn't agree with you more. We developers are the absolutely worst people in the world to QA our own projects, as we develop tunnel vision towards how the software is used; and we develop an unconscious aversion towards doing things that will cause problems for our systems.
I test my code extensively during development, but am still absolutely amazed at how quickly my customers find errors that didn't occur during my testing. My workplace can't afford QA staff, so we developers must perform development and QA work. It works for us solely due to our customer structure, but it would be a nightmare in most other contexts.
QA is a cost center, management is always looking to cut or make more efficent cost centers (and at the same time maximize profit centers)
It's no wonder every manager under the sun thinks they're the genius who can "solve the QA problem" and return 1-6 people's worth of payroll to the bottom line for that big annual bonus. The problem is that while YES you can get away with no QA in a very high functioning development team, on certain products,
A) most teams are not high enough functioning, even if they think they are to get away with no QA for 6+ months before quality fades
B) usually only a few developers actually have contact with the customer and in particular, for enterprise-level apps (i.e. the software that drives people's business) these things are super complex and probably your customer depends on a particular feature working a particular way as a critical part of their workflow, and if you change that both of you are screwed
So yeah, your company functions ok doing facebook for dogs, snapchat for cats, or XYZ consumer web/mobile app? Cool. Your product is a button that has some business logic attached to it, yes you can totally QA a single function app. Enterprise apps that people actually use and depend on to function the same from release to release generally need dedicated QA that understands all the niche use cases that makes the product so popular in the first place.
moox. for a new generation.
This is really a lame-brain position. As a long time developer I have two mottos:
* If it isn't tested, it doesn't work.
* A developer make assumptions all the time.
I can guarantee that, if I was responsible for QA on my own code, things would be broken in subtle ways.
Many years ago I did QA consulting for AT&T. We set up a system where the human factors engineering group would document an application in a specification. One copy went to the developers and one copy went to the QA personnel. The QA staff designed a test plan from this spec and then implemented it against the product sent from the developers. I can't tell you how many things the developers missed because of assumptions they made that were caught by the test plans.
Yes, as it's become fashionable to get rid of non-programmer testers, software has gotten worse for non-programmers. (Disclaimer: I'm a programmer/tester.)
The Iconic's idea is hardly new. Yahoo tried it during the Mayer era (https://slashdot.org/comments.pl?sid=8469165&cid=51105569); how'd that work out?
QA/QE/whatever you care to call it is critical when doing anything important. They provide an independent analysis that the job was done right. Not only in code, but in pretty much any industry. Creators fall in love with whatever they are creating, and think it's perfect. QA should provide a more objective view. Just this week, I was editing some code. I'd tested it and my tests were fine. Send it for review, one reviewer says "why did you change this". Turns out, I'd changed something in an area I didn't mean to. It would have been caught when it went into production, but then a giant flaw would have made it into production.
On who's going to do QA.
If she's a QA professional, then it's ok.
Otherwise do it yourself to save resources.
Sent as ripples into the electromagnetic field. No single photon has been harmed in the process.
If you are standing up some trivial web application and call it your 'software project', then maybe you are overthinking it to have a lot of people in different roles. Of course if your web application is more involved or more severe, then you need testing from a separate perspective.
If you are writing some particularly custom software for a particular market, then you should probably have a pool of testers that may not know how to program, but they are probably more in tune with your target userbase. I once worked in a copmany doing CAD/CAM software, and the software developers roughly knew what to do, but we hired mechanical engineers to do testing because they would call us on UI BS or make suggestions that are relevant to users.
XML is like violence. If it doesn't solve the problem, use more.
Developers can add assertions and write unit tests that can test conditions that should never reach a particular module. That should be responsible for producing unit tests and running all tests as a regression suite. The best team I worked for had a policy that when a bug was reported you wrote a unit test to reproduce it at the lowest level, then fix it. This test was added to the test site and solid to every future version. However this is not sufficient, a QA team needs to test the resultant project. It's possible that the developers are not looking at the big picture, or even have misunderstood the requirements and written excellent coffee to do the wrong thing. It's also possible that other chances may interact in unexpected ways, so a whole release needs QA testing.
Peer review is great. I introduced it at my company. It's not QA.
Users do stupid shit, shit no programmer in their right mind would ever think of doing. As an extremely simplistic example, at my job we have a script that calls nap with various arguments, including a list of IPs provided by user. The programmer tested it with some IPs. The peer could test it with some other IPs. The programmers made sure it works. Users will pass it 0.0.0.0/0, and fa70:638:363:7363::76. QA does the same, because QA is trying to break it. Programmers ensure it works, QA figures out how it can be broken (and are therefore an important part of security).
Aside from that difference, excellent programmers (10% of programmers) will PRODUCE quality. QA will ASSURE quality. With programmers testing, you can hope the quality is good. With QA testing, you can know the quality is good.
I've been on both sides of the Dev/QA divide. I'm strongly in the camp that developers should be responsible for writing automated (possibly unit) tests of their own code. However, before the code is shipped to the customer QA should have a go. It is way too easy for a lazy or even well meaning developer to ship something out of spec. Having the developer do all their own QA is like letting the prosecutor also be your defense attorney. It's patently and obviously wrong.
"If our site is down now, people will generally come back later,"
They don't care about quality and how it affects their customers. They think their customers have all sorts of extra time to do double work because of software errors. I have to wonder if they company takes the same lackadaisical approach to the quality of their products? ~It's OK if our products are bad, because the customers will just buy another...~
This is the quality of QA you get when you have your dev team do all the testing.
https://blog.codinghorror.com/...
When does Bob start writing horrible code, knowing that Margareth will find the bugs and clean it up for him? And when does Bob get the promotion because his code shipped faster, and Margareth's slower developed code fails to be presented early? So Bob gets to go to luch with the boss, go tot the company picnics, and gets invited to design review meetings because "they ship faster"?
I've seen this happen. It's been done to me, as "Margareth" in the story. I eventually had to talk with my manager, from quite a long time ago, and warn that manager what was going to happen to Bob's workflow and code quality when I went on vacation.
You pay QA people less than you pay devs and it's a different skillset.
So this guy is wasting money and getting a worse product. So win/win.
Developers should be responsible for making sure their code is run through static analysis, that it has adequate unit tests (that pass), that automated integration testing is in place and passing, that peers have reviewed the code, and similar 'best practice' sort of things. They should also be engaged in using the product and so on, and if management wishes to spend their time this way, testing OTHER developers code.
A developer testing their own code outside developing and maintaining automated tests is a monumental waste of time.
There is no ambiguity here. The software that allows control characters or any other non-alphanumeric characters is the software with the bug, since by your own admission the spec never explicitly states they should be allowed. For extra extra credit, if the spec said they were allowed then it would be the spec that has the bug in it.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
(OK, I found one occurrence.) But seriously, this is what's being sold in management consulting "corporate DevOps transformation" packages. The first rule is almost always, "Go fire your QA department and make your developers write automated tests." This is exactly what the executives paying the management consulting firm for their "digital transformation strategy" want to hear, and that's magnified by a bunch of Google "visionaries" touting the same. Not every business is Google, and IMO DevOps works only when you can swap out everything, not just pieces.
If QA is just mindlessly pushing buttons and seeing what breaks, maybe that makes a little sense. But, I do systems integration work, meaning our team gets to figure out how hundreds of different modules from various dodgy offshore contractors fit together. I'm of the opinion that devs doing their own testing works well when there's little component interaction and the consqeuence of failure is a 400-series or 500-series HTTP response. When you're talking about a system made up of tons of components, no matter how microservice-y you make ti, there are still plenty of opportunities to introduce issues, and someone needs to be able to catch them.
" the company has now moved all of its QA workers into engineering roles."
So inexperience engineers are going to be releasing code with no QA.
You can't make this stuff up.
My projects were always small. I was usually the only developer. Sometimes there were one or two others
I always wanted independent testing, in fact, one of my favorite rants was.. "the worst possible tester is the person who wrote it"
But, independent testing never happened, so I did my best. I started writing automated test tools long before they were popular
Of course developers should test their own code by running it in the debugger and making sure they're not checking in obvious garbage.
And of course developers should do unit testing modules to verify the correctness of certain libraries and submodules of their code. This correctness helps to make sure submodules behave correctly.
But you need a separate QA team. And you need two types of testing: directed testing (with specific testing instructions to test components of the software) that are put together by a QA manager. And you need undirected testing--basically QA folks just screwing around with the software.
There are several advantages of having a separate QA team. First, QA testers do not have to have the same level of experience as software developers, and so you can hire testers with less experience and for a lower salary. (If you have a QA manager who writes good test instructions, you can pretty much hire damned near anyone to test the software. I mean, presumably if your software is used by any random person off the street, you should be able to hire any random person off the street to test, right?)
Second, developers often wind up concentrating on a few modules when adding features or fixing problems, and so often fail to do adequate integration testing across the product, looking for unexpected side effects. (In essence, the biggest problem are those "unknown unknowns" which cannot be found unless someone just farts around with the software.)
(And don't tell me all this should be automated. That sort of automation is basically front-loading all the possible testing you may want to do--and completely ignores the institutional knowledge of your product gained by QA testers playing with the product. In the past when I've gone to work for companies with a well staffed QA team, I first go to the QA team, not to the development team, to understand how to use a product.)
In general I'm a big believer in the model of software developer as a Knight surrounded by support staff. Your developer is probably the most expensive person on your payroll--so why waste an expensive resource on stuff that can be offloaded to people who can probably do various support jobs more efficiently for a lower salary? I know that sounds a bit brutal, but there is definitely an efficiency argument for software developers not writing business requirements, manning customer support lines, or doing customer sales--so why the hell do we expect developers to take over testing?
> Fashion retailer The Iconic is no longer running quality assurance as a separate function within its software development process,
You'll be sooorrrrreeee.
...who tests his own code as the final authority is a fool, claiming omnipotence that is quite common to the breed.
To be sure, I now--in retirement--only write code for my own needs. But, when I was programming as a profession, I accepted the fact that I am not omnipotent. I have "blind spots" in my knowledge, I have unknown things about which I've never thought before. Just because the language is as simple as BASIC doesn't mean that the writer of that code doesn't have blind spots in logic or comprehension...or even understanding of the customers' needs.
Bypassing a) The writing (and vetting) of a formal specification, and b) independent testing...may lower the cost of programming, but it has the hazard of completely overlooking something that is obvious ONLY after you've triggered the defect....as Equifax recently found out, to their financial peril.
When I program for somebody else, I require they understand the costs in both time and money of a) Writing a comprehensible specification, and b) Independent testing to ensure compliance with that specification. The extensive of each of these is dependent upon the risks at stake. If it's a personal novelty or utility, I can let these rules lapse, because I'm "evolving" code. But one cannot "evolve" code by subjecting it to test by the end-customer. That way lies frustration and ultimate dissatisfaction.
In my experience, the best programs written are developed by teams made up of members with (intentionally) different viewpoints and experiences, who respect--yet challenge--each other. I can recall, in the early 60's (IBM 709 and 1401 days, for you ole timers) when I was part of a team developing one of the first "shareware" packages for the 1401: We critiqued, debated, evolved and distributed a robust program that was a platform for quick application development. It was a delight, and when I happened to be in Australia, eight years leter, tripped across a team using our shareware as the basis for other application programs...and they'd never experienced a failure because the package (and the documentation) had been vetted by so many experienced (yet divergent) people.
If you debug all your own code, and are the sole party declaring it "Good and Finished," you are deluding yourself. None of us are quite that perfect, especially in anticipating situations or environments which we've never imagined or experienced.
Someone who wrote something will overlook many errors that they wrote.
As for writing tests -- you write tests for the edge/corner cases that you know of in such code. You don't write tests for the bugs you don't know about.
Some testing can and should be done by authors, but no more than a minority of it. Of course, who tests for "usability" or "user friendliness"? Can't ever have a SW engineer testing those things -- hard enough to get them to fix the problems, let alone call them problems in the first place.
Ofcourse coders should test their own work (it amazes me how many coders don't even do a simple check on changes when checkin their code in), but there should always be a tester who tests the working of the changes, ALWAYS..
Developers are by definition expert users of the software they write. This leads to a usage pattern that tend to leave certain bugs unexposed. Typically bugs related weird usage patterns etc. To find such bugs you need a certain level of stupid that only real users can supply.
Part of me thinks It shows a remarkable lack of understanding, and part of me thinks it could be very insightful.
My first thought was that its already very well understood to be a terrible idea to have only the same person who wrote the code production test it too (because they make all the same assumptions in testing that they also used to write it, so miss many test cases). Also that company is now putting high-cost developers on testing that could be performed by cheaper employees. But then.... maybe developers will actually self-improve quicker and even self-select if they now have to be personally responsible for writing bad code and don't have a QA department to hide behind.
We were a big company. Every developer was responsible for unit testing of their software. It would only go to QA for system-level testing. If it failed, it went back to the developer to be fixed. We had a VERY low failure rate as a result! FWIW, I was the one who instituted this process.
Sometimes, real fast is almost as good as real-time.
Longer answer, the developer is given requirements, then they code to their understanding of the requirement, and if tested by the same developer they will only confirm the software meets their understanding of the requirement.
Having a separate QA/Test/Validation that reads the same requirements, forms their own understanding of the requirements, and tests to that understanding... if there is a difference, the tester an developer will refund their joint understanding and produce the desired product.
Granted, selling fashion is not a life or death proposition, but they should want the code to match the specifications, and having the developer test the code themselves is far from optimal - they have a bias regarding the software.
I tested software in clinical drug trials, and we would never just 'take the developer's word' that the code was correct, and neither would our clients or the FDA.
Ken
So, you see the problem.
https://biblipole.com/Funny-St...
That's a humorous image, but when you think about it, it shows how the psychology and rewards for testers are different than for developers.
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
Customers charge a hell of a lot more to do QA testing than the regular QA people do! Think about it! ;-)
"If Engineers built buildings the way Programmers write programs, the first woodpecker that came along would destroy civilization!"
I'm always amazed how stupid and naive these people are that believe this. Developers need to QA the things a separate, dedicated QA tester can't easily. The QA guy needs to test from a customer perspective, finding new and imaginative ways to be stupid. My takeaway is that your developers are cheap and shitty. Fuck, just testing on a full blown, high-core, high memory PC will often hide bugs found on cheap office PC's not running SSD's and shit. Full disclosure, I primarily work in QA, and routinely complimented on my ability to find bugs and break things in ways the developer couldn't.
You can't trust the same eyes that made the mistake to catch it.
It's not even a matter of being ethical or scrupulous, it's a limitation of awareness. A developer who coded a logical mistake could be situationally blind to it.
Let someone else do it.
LK
"Hi. This is my friend, Jack Shit, and you don't know him." - Lord Kano
I hear about test driven development, and all the tools being put out to allow unit tests and easier integration and regression testing. I suppose it didn't make sense to have another lay in the development cycle?
"...whenever any Form of Government becomes destructive...it is the Right of the People to alter or to abolish it..."
Development and testing are sure as f*ck two different tasks. But well made code should already have a well rounded set of tests, to ensure absence of regressions and to prove that new features do work as intended. A damn huge part of coding is ensuring that your new pretty pile of code does work.
So as a software engineer who has worked for 3 years in a team with an integrated tester, I can tell you that we can learn hugely from QA people, how to make your code robust, testable, provable. And QA can learn from devs too; QA should not be mayhem monkeys, they should spend time carefully crafting new edge tests and not do this robotic process of testing that is boring to death. Death by manual regression tests could be a kind of capital punishment in some high tech countries.
Software Engineers out there, you should try and spend some time with your nearest tester and pair-test (yeah, just like pair-programming) with him. You will learn soooo much, that is guaranteed.
Stupidity is the root of all evil.
Yes your developers should be during QA. No you should not be stopping QA outside of developers, that is just stupid. You need both.
Recently we've forced developers to start doing some additional QA. The reason? Lack of code accountability.
I've gotten to test and QA code so bad, that it barely runs, or doesn't even load, or is so riddled with problems one has to wonder how it was they even did the coding in the first place successfully without having to run into so many problems that it would prevent them from coding other parts of the application. We would spend so much time correcting obvious errors, and just getting the application to "work", that there is little time in the project for more detailed QA in regards to if it is meeting actual business requirements, or subtle specific case conflicts etc... When at the end of it and you are running out of project time, and you start pushing back at the developers, they push right back saying it is all the fault of a lack of QA. The solution is that they are forced to QA themselves, and to demo proving that what they have accomplished at least on the surface is a working product without any obvious problems *before* any other QA type work is done by anyone else. As mentioned I've seen stuff that is so bad that I'm like "did anyone look at this at all ever? How did then even code it with at least running the application at least once? Did they actually compile and test their code at all, or just write it all free hand and cut and paste the code in, wiping their hands and exclaiming, DONE!"...
At any rate, only having developers is crazy, and I foresee lots of problems. Typically developers might have a vague understanding of the business, but they are usually just given the requirements documents and their marching orders. The should be able to catch obvious coding mistakes, and some process stuff, but realistically they are not going to know enough to do all the QA themselves unaided.
I've been the programmer and the engineer and the QA tester. And this all boils down to the quality of each. I remember when I was the programmer/engineer on a long standing application. Management brought in a QA person, who was basically an intern who was tasked with finding bugs (probably because someone complained the app was "buggy" when it was really just overloaded and being used outside of its design parameters.) This guy comes to me with a laundry list of bugs, and the biggest one was where the application crashed when they were not on a specific tab, and the user clicked a certain button. He thought he had found the holy grail of bugs. He was very excited. His excitement turned to dismay when he asked when I would be fixing this bug. I told him "never". He was incredulous. Why? Don't you see how bad this is? I asked him how many times the bug had been reported. He said that it had NEVER been reported. HE had found it! And I explained that a user would never push that button without being on the tab it was associated with, because they would want to see the quote that was being sent out and it's amount before submitting it. So it was a VERY low priority and I wouldn't be doing anything about it anytime soon. OTOH, I was a QA tester for a medical product at one point. I developed the test cases and executed unbelievably complex test scenarios (largely because I suffered from a disease this product was used to help manage) and so had a complete "user" mindset. My documentation on it was formidable as it all had to be made available to the FDA. So this was some very high stakes testing, and I was able to uncover many bugs that would have sunk the product had I not been as thorough as I was. The company that engaged me on this project was overwhelmed with just how detailed I had been, and turned to me again when they came out with another similar product. I declined that engagement as I had moved onto a full time gig instead of consulting, but I probably could have made a good living with just them as my only client had I chosen to. Bottom line on all of this is that it depends on the people. Quality people, regardless of role, is what will make or break Quality Assurance in products and software. If you have slipshod, lazy programmers, no amount of QA will fix that. If you have slipshod, lazy QA engineers, no amount of QA will fix that either. Where the problem lay in this case is hard to determine. But my guess is that this org has a problem in both areas, which is a reflection of their management more than anything else.
Brawndo: It's what plants crave!
You are a bad coder!
The big try catch is so you log the unexpected errors. Not to trap the ones you know how to handle and recover from.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
We should do our own Unit Tests.
Someone else should monitor code coverage, cylcomatic complexity, etc., and push us for more and better tests.
Someone else should do QA.
Someone else should write system integration tests (where you test the installed system, not an isolated piece of code)
I think it's good that the developers start taking initiative and thus be able to add the type of quality that the client needs, in my case I develop websites with a standard that I was doing depending on what the client wanted. for example my website Zona viajes
It may have been Fred Pohl who said "The problem with being your own editor is you have no editor."
There's no time like the present. Well, the past used to be.