LinkedIn Is Open Sourcing Their Testing Frameworks (github.io)

destinyland writes: LinkedIn is open sourcing their testing frameworks, and sharing details of their revamped development process after their latest app required a year and over 250 engineers. Their new paradigm? "Release three times per day, with no more than three hours between when code is committed and when that code is available to members," according to a senior engineer on LinkedIn's blog. This requires a three-hour pipeline where everything is automated, from committing code to releasing it into production, along with automated analyses and testing. "Holding ourselves to this constraint ensures we won't revert to using manual validation to certify our releases."

Comments

Arbitrary rules

[Anonymous Coward]

Why 3 hours? Why not 2? or 4? Why not manual? Bugs are by their nature UNanticipated and automated testing tools only test anticipated bugs.

"Holding ourselves to this constraint ensures we won't revert to using manual validation to certify our releases."

Why? It seems to defy logic. The first time you do user testing is when real world users are using it. That's moronic.

Verification and validation

[DeBaas]

First of all, thank you Linkedin for open sourcing this! Always good to share.

First, three hours is not enough time to conduct any manual testing steps, so holding ourselves to this constraint ensures we won’t revert to using manual validation to certify our releases.

I've been in testing for some time and have been taught to make a distinction between verification and validation.
Verification is checking if the software works according to specs. Validation means: does it actually work for us. By defintion that means that you can automate verification but not validation.
Is that just semantics? Not for testers. In the test community there currently is a big debate going on checking vs testing.
See i.e. Michael Boltons blog. . Checking can be done automated. Real validation in my opinion can not.

What I am curious about is the 'to production each three hours' That sounds great, but although I don't use a linkedin app on my phone, I am still pretty sure users don't get their app update three times a day.. With such rapid deployment, I suspect it takes multiple deploments before it adds up to a significant increase in usable functionality.

Many small releases means in general that mistakes are also small and quickly fixed. I am actually in favour of them. But it is not a full garantuee that one of these small releases will not break something badly that would have been found by even a limited manual test. The chance that that happens may be much lower with small releases but it still exists and the impact is still high.

Automated tests can perform a huge amount of checks quickly. Humans can't beat that. But they can also overlook the blatanly obvious. I would hope they would have manual testing at least prior to releasing new functionality. To find these things, but also to do some validation by the definition above.
Else I suspect it may work well for them for quite some time but it may bite them badly at one point as well.