Slashdot Mirror

First of all, thank you Linkedin for open sourcing this! Always good to share.

First, three hours is not enough time to conduct any manual testing steps, so holding ourselves to this constraint ensures we won’t revert to using manual validation to certify our releases.

I've been in testing for some time and have been taught to make a distinction between verification and validation.
Verification is checking if the software works according to specs. Validation means: does it actually work for us. By defintion that means that you can automate verification but not validation.
Is that just semantics? Not for testers. In the test community there currently is a big debate going on checking vs testing.
See i.e. Michael Boltons blog. . Checking can be done automated. Real validation in my opinion can not.

What I am curious about is the 'to production each three hours' That sounds great, but although I don't use a linkedin app on my phone, I am still pretty sure users don't get their app update three times a day.. With such rapid deployment, I suspect it takes multiple deploments before it adds up to a significant increase in usable functionality.

Many small releases means in general that mistakes are also small and quickly fixed. I am actually in favour of them. But it is not a full garantuee that one of these small releases will not break something badly that would have been found by even a limited manual test. The chance that that happens may be much lower with small releases but it still exists and the impact is still high.

Automated tests can perform a huge amount of checks quickly. Humans can't beat that. But they can also overlook the blatanly obvious. I would hope they would have manual testing at least prior to releasing new functionality. To find these things, but also to do some validation by the definition above.
Else I suspect it may work well for them for quite some time but it may bite them badly at one point as well.

Mod parent up.

http://www.developsense.com/bl... is a treasure-trove of testing (and other) information. Simply reading his (and similar) blogs is an quick, easy, and effective (and free!) way to learn about testing. Also, be sure to check out the blog of James Bach for the same reasons: http://www.satisfice.com/blog/.

"After complaining about click-baiting laden titles, in your own writing at the link you provided, that's a pretty ironic statement." It's not a statement. It's a question, rhetorically delivered, that addresses a legitimate concern. Here's a fun game for the whole family: go to LinkedIn, and do a search for the string "29119". Note the preponderance of consultancies that offer services in interpreting and explaining 29119. Have a look at the constituency of the working group; note the overlap between those companies and those who are enthusiastic suppliers to the ISTQB certification mills. Have a look at the minutes of the meetings of the working group, and look for phrases like "marketing the standard" (rather than, say publicizing the standard). Are you really trying to claim that your business and yourself have no financial motivation in your actions? Or do you do all of your consulting gratis, merely on principle alone? That's another FAQ. “In one sense, it won’t make any difference to my business if 29119-1, 29119-2, and 29119-3 are left to stand, and if 29119-4 and 29119-5 move from draft to accepted. Rapid Software Testing is about actual testing skills—exploration, experimentation, critical thinking, scientific thinking, articulate reporting, and so forth. That doesn’t compete with 29119, in the same kind of way that a fish restaurant doesn’t compete with the companies that make canned tuna. We object to people manipulating the market and the ISO standards development process to suggest to the wider world that canned tuna is the only food fit for people to eat. I discuss that here: http://www.developsense.com/bl... “In another sense, 29119 could be fantastic for my business. It would offer me a way to extend the brand: how to do excellent, cost-effective testing that stands up to scrutiny in contexts where some bureaucrat, a long way away from the development project, was fooled into believing that 29119 was important. At the moment, I’m happy to refer that kind of business to colleagues of mine, but I suspect that it would be something of a gold mine for me. Yet still I oppose 29119, because what’s in my interest may not be in the interests of my clients and of society at large. “Let me be specific: There are existing standards for medical devices, for avionics, and the like. Those standards matter, and many of them are concise and well-written, and were created by genuine collaboration among interested parties. Testers who are working on medical devices or on avionics software have a limited number of minutes in the working day. As someone who flies a lot, and as someone who is likely to require the help of medical devices in the foreseeable future, I would prefer that those testers spend as many minutes as humanly possible actually investigating the software, rather than complying (authentically, pathetically, or maliciously) to an unnecessary standard for process modeling, documentation, and strategizing (a standard for developing a strategy—imagine that!). "You are free to ignore this standard" Yes, of course I am... until it creeps into regulation as the NIST points out in the second-last paragraph here: http://www.nist.gov/standardsg... ---Michael B.

That's certainly an important point. There are others. To me, the issue not just the cost of preparing the documentation, but the degree to which compliance with the standard displaces the goal of actually testing the product or service. A moment that a tester spends on useless documentation is a moment in which she's not focused on identifying risks and finding problems that would cause loss, harm, or annoyance. http://www.developsense.com/bl...

Would you agree that consensus gained by attrition and rent-seeking should be used to determine the way things are done in health-, safety-, or finance-related enterprises upon which you, your loved ones, and your nest egg depend? Are you saying that IEEE/ISO working groups should ignore what's going on in the world because the process is designed to favour those who are driven by profit, but who do not have skin in the game? http://www.developsense.com/bl... ---Michael B.

That is both an advantage AND a disadvantage, since it affords the potential for rent-seeking (http://www.developsense.com/blog/2014/08/rising-against-the-rent-seekers/ ). Rent-seeking may lead to wasteful documentation, compliance vs. competence, and other forms of goal displacement. See also http://www.developsense.com/pr....

Long story short, VSS is one of the worst piece of source-control software ever. This is a quite beaten to death horse, so here are 3 articles on the subject: Visual SourceSafe Version Control: Unsafe at any Speed?, Visual SourceSafe: Microsoft's Source Destruction System and Source Control: Anything But SourceSafe.

I admit, setting up a CVS client can be a bit of a pain the first time. But I set up a CVS server in no time at all. It was all very simple and I'm not that great with Linux. As for reliability, I've used CVS for years and I have NEVER had any sort of corruption with it whatsoever. I have yet to have a SourceSafe setup that didn't get corrupted at least once. God forbid you want to do anything beyond check-in and check-out and you're practically guaranteed corruption.

SourceSafe IS that unreliable. Always has been, always will be. The base code is crap, pure and simple. Oh SourceSafe how dost thou corrupt thyself? Let me count the ways! Oh wait, I don't have to. Michael Bolton already has. (BTW, I suspect he's not the singer.)

http://www.developsense.com/testing/VSSDefects.htm l

This sounds like it shares some basic principles with Pairwise Testing. Basically, the theory here is that a large percentage of bugs come in through the combination of two inputs. So, if there are 10 bits in a flag, you need to only make sure that each pair shows up once: with various constraints, I got down from 1024 to 12 values to check on a recent test.

Check out http://www.developsense.com/testing/PairwiseTestin g.html for a better explanation, or anything else Google brings up for you.