Slashdot Mirror

← Back to Stories (view on slashdot.org)

No More QA: Yahoo's Tech Leaders Say Engineers Are Better Off Coding With No Net (ieee.org)

Posted by timothy on from the who-ya-no-longer-gonna-call? dept.

Tekla Perry writes: A year ago Yahoo eliminated its test and quality assurance team, as part of project Warp Drive, its move to continuous delivery of code. The shift wasn't easy, Yahoo tech execs say, and required some "tough parenting." But the result has been fewer errors because "when you have humans everywhere, checking this, checking that, they add so much human error into the chain that, when you take them out, even if you fail sometimes, overall you are doing better." And the pain wasn't as great as expected. Yahoo's chief architect and SVP of science and technology discuss the transition.

12 of 216 comments (clear)

  1. Sounds like an MBA plan! by Dunbal · · Score: 5, Insightful

    We were tired of being constantly bogged down by all these mistakes and bugs, so we got rid of the people who kept telling us about all the mistakes and bugs. Now our code is 100% mistake and bug free! Next step, get rid of our expensive experienced coders and replace them with cheaper outsourced coders with "equivalent" experience. We'll save so much money what could possibly go wrong? And the third and final phase of our plan is that in order to motivate our coders we will be paying a bonus that scales with the amount of code written. The more code you write, the better the bonus!

    Don't you just love management?

    --
    Seven puppies were harmed during the making of this post.
    1. Re:Sounds like an MBA plan! by Junta · · Score: 4, Insightful

      The argument is actually pretty bad. By all means add more automated testing and hopefully make the QA process smoother, but skipping it entirely (they fired the QA team) is a recipe for disaster, and the only thing they have to measure the effectiveness of this is the number of issues in their issue tracker, which was *fed* by the QA team. It's like a company firing all their warranty people and bragging about how their warranty claims have gone to zero. The filed issues may be lower, but that does not speak to the final product.

      Testers don't always follow proper testing procedure

      This is a double edged sword in my experience. The testers are using the software much like a human would, and they do unexpected human things like the end user would be doing. This particularly helps identify usability issues, which isn't really an automation sort of thing. It's also a lot of fuzzy stuff that is usually not specifically laid out in the spec.

      robustness towards matching the spec

      Sometimes single minded march toward matching the spec is a bad idea, because the spec is bad and needs to be revisited. Sometimes this plays out in development, and sometimes this plays out in QA when humans clearly have a harder time with the interfaces than expected. Also generally speaking for me a spec is pretty high level, open to interpretation, and refined throughout the process as the details of the implementation get fleshed out.

      --
      XML is like violence. If it doesn't solve the problem, use more.
    2. Re: Sounds like an MBA plan! by rgmoore · · Score: 4, Insightful

      It was up to me to make sure my own code was quality enough before releasing it, and that aspect terrified me enough that I did learn to write quality code (which basically means you are testing your own code thoroughly, doing your own QA).

      The problem with testing your own code is that you're likely to miss entire classes of bugs. You can be very effective at catching the kinds of bugs you can think of, but those are always the easiest bugs to catch in the first place. The tests you write for yourself will never do a good job of catching errors in your assumptions about what the code should do, what kinds of inputs it needs to handle, etc. Catching those kinds of conceptual bugs really requires adversarial testing from somebody who isn't starting from the same set of assumptions.

      --

      There's no point in questioning authority if you aren't going to listen to the answers.

    3. Re: Sounds like an MBA plan! by phantomfive · · Score: 4, Insightful

      Yeap. That's where pushing to live production is going to be your brutal tutor.....if you have cognitive biases that are hiding bugs from your psyche, then the system will show you to them (most likely in the most inconvenient time), and you will learn. It's harsh.

      You can get the same experience (but with less harshness) by giving yourself the goal to never let a bug get past you to QA. Then QA will be a kinder tutor that helps you learn.

      --
      "First they came for the slanderers and i said nothing."
    4. Re: Sounds like an MBA plan! by NormalVisual · · Score: 3, Insightful

      I can give a counter-idea: in my first job, I didn't have QA. It was up to me to make sure my own code was quality enough before releasing it, and that aspect terrified me enough that I did learn to write quality code (which basically means you are testing your own code thoroughly, doing your own QA).

      I think it's always necessary to test your own code via unit tests and whatnot prior to letting someone else have at it, but I think it's also important to have that second set of eyes on it from a behavioral standpoint. Often we're too close to the code to really beat it up properly and subject it to really weird edge cases, and good QA people are quite adept at doing that. What I've found works pretty well is having the code go through QA, but still being held to some kind of personal accountability for it as well. Even if it's nothing more than your co-workers laughing at your QA bug reports and saying your code sucks, it's some more motivation to get it right the first time, and if it passes QA you have at least some degree of confidence that it works properly.

      Another thing I've learned is that the stereotypical animosity between dev and QA is counterproductive. It's much better to think of them as two parts of a team working toward the same goal, and it's beneficial to foster an environment where the devs actively encourage QA to break stuff instead of trying to blow them off all the time.

      --
      Please stand clear of the doors, por favor mantenganse alejado de las puertas
    5. Re: Sounds like an MBA plan! by phantomfive · · Score: 4, Insightful

      Another thing I've learned is that the stereotypical animosity between dev and QA is counterproductive.

      Yes. QA is absolutely on your side!! Every time they find a bug for me, I thank them profusely, because better them than a customer.

      --
      "First they came for the slanderers and i said nothing."
    6. Re: Sounds like an MBA plan! by Anonymous Coward · · Score: 2, Insightful

      Call volume is unusually high. You are number [37] in line. Please be patient and listen to the pleasant music for approximately [109] minutes. [buzz buzz brrrrff buzz ggggzhxx buzz buzz ...]

    7. Re: Sounds like an MBA plan! by phantomfive · · Score: 5, Insightful

      Do you think that users' data security is the proper object of harsh lessons for coders?

      If you are depending on QA to make sure user data is secure, you're going to fail. That's not what QA does, and hacking is rarely in their skillset.

      When companies start caring about security, they usually start a separate team "Red Team" or something whose entire job is to try to hack and find security holes in the project. (I would argue this is not sufficient, because companies who have done this still fail. If you want secure software, it has to start from the beginning. You need to teach your programmers how to write good code, and security has to be a priority on every line of code that is written. Security can't be tacked on as an afterthought).

      --
      "First they came for the slanderers and i said nothing."
  2. You can't make this stuff up by rudy_wayne · · Score: 1, Insightful

    A year ago Yahoo eliminated its test and quality assurance team

    The perfect behavior for a company that is worth less than zero. (Subtract their shares of Alibaba from their market cap and you get a negative number).

  3. Do they know what QA is? by Razed+By+TV · · Score: 5, Insightful

    If your QA people are adding to the problems, you are probably doing it wrong.

  4. Oh, they re-invented Test Driven Development. by mikelieman · · Score: 3, Insightful

    >It has also, he said, forced engineers to develop tools to automate the kinds of checks previously handled by teams of humans. An engineer might go through an arduous process of checking code once—but then would start building tools to automate that process.

    I suspect they're too dumb to realize this and just tell everyone, "We're saving money and delivering better code using TDD"

    --
    Technology -- No Place For Wimps! Grateful Dead and Jerry Garcia Chatroom -- http://www.wemissjerry.org
  5. Re:Awesome Idea! by TWX · · Score: 3, Insightful

    They still do a lot of end-stage testing though. Pressure testing, structural testing, failure-mode testing, load testing, load-shift testing, strike testing, and probably all other manner of tests. They do perform a lot of tests as they develop but that's no replacement for the final-stage testing that confirms predicted behavior or verifies that systems integrate as expected.

    To pretend that there's no end-stage testing is simply ridiculous.

    --
    Do not look into laser with remaining eye.