Testers Tower

The “Test is Dead” story is not news anymore. A lot of the good testers out there have since tried to reframe the idea, to put in into context. See Mark Tomlinson’s blog post on the subject or listen to this TWiST podcast (requires basic membership; it’s free) for good examples.

To refresh your memory, here’s a short recap:

After posting an ominous article about the death of testing this June, Alberto Savoia finally got to explain those cryptic lines in his opening keynote for GTAC 2011. In short, he believes that building the right product – the right “it” – is far more important that building “it” right. Or, to put it another way:

Testing slows you down… by the time our competitors had a well tested product, we had millions of users.”

So he appears to claim that it is far more important to test the concept than the actual product. He even wrote a book about it.

I have to admit, I can see a lot of potential in this approach. The success of twitter is just one example of how users can easily overlook even serious bugs in the product as long as the “it” part feels like warm apple pie.

Then again…

Going by Scott Berkun’s awesome guide of how to call bullshit on a guru, let me ask this:

“When is the theory you are advocating wrong?”

Well, off the top of my head…

• Online banking – I won’t trust you with my money unless you pen-test the crap out of it
• File hosting – I won’t trust you with my files unless your service has 99.9% up-time and is secure
• Email – again, has to be accessible 24/7 and secure.
• Using credit cards – buying stuff online/paying for services
• Cars – Yes, cars use software too.
• Parliamentary elections – don’t laugh, read
• Prisons – just because it happened.

…
And I haven’t even mentioned airplanes, medical devices or military computerized weaponry.

Still. In the context of getting the next big “it” to market in Silicon Valley… yeah, sure. I cannot deny that it worked out really well for twitter. I’m pretty sure it will work for a lot of awesome new software in the future too. In their context, being on the market will probably far outweigh the risk of any and all bugs.

So is testing dead or not?

To quote Scott Barber’s hilarious mock interview blog post on the same subject:
“No”.

Silicon Valley is not the only place where development is done these days; neither are awesome social networking solutions the only kind of software in existence. Heck, a lot of software doesn’t even HAVE a market, only a customer that is already paying for it anyway and wants his product to actually work once it’s done.

The examples above are just the tip of the iceberg. Most of the software I see in my work has to be tested, no matter what. As far as I am concerned, it’s only when the health, money and time of the customers are not at risk that a “test is dead” approach can be considered.

I could go on, but Tomlinson’s article covered the test-is-dead bashing part well enough already.

However, there are some aspects that haven’t been discussed as of yet.

Taking it further

Now: why is it that when Savoia gave his keynote, a great number of good people out there deemed it necessary to reframe his ideas?

This is a prelude to a longer series prompted by the test is dead issue and the general problem of saying something that people are bound to misinterpret. More to come.