Testers Tower

The problem of talking about testing

I tend to read a lot of books, blogs, forum posts and tweets related to software testing. One of the things that I keep seeing time and time again is the myriad of contradicting opinions people have about certain aspects of testing. The eternal “To automate or not to automate” question and more recently “test is dead” are both prime examples, but there are others too — whether testers should learn to code or not, the use and misuse of metrics, the advantages of one tool over another, the value of certifications, scripted test cases and test documentation… the list goes on, but you get the point.

So why is it that opinions differ so widely?

I have been pondering the subject for a while now, and here is what I came up with:
It’s probably because context matters. A lot.

I call it the context problem of talking about testing. The context problem has a chance to emerge whenever someone says something specific about software testing that is possibly true in his context, but might be a really bad idea somewhere else.

The whole Test is Dead issue is a prime example of a context problem of talking about testing; it’s also what started me on this subject. If you are not familiar with the phenomenon, read about it here.

The problem of saying something that people are bound to misinterpret

Unlike some other blogging testers out there who have written about the subject, I do not know Alberto Savoia personally. I even have to admit that before the whole hype happened, I was not familiar with his work at all. However, unlike some others, I do not see reason in this madness. I don’t think that how he acted was acceptable as a professional. I fact, I don’t think it was acceptable at all.

What I’m worried about is that somewhere, someone in a management position is going to decide to get rid of all the testers and hire more developers instead, because someone said this is the way to go now. I’m worried about the fact that people who are not intimately familiar with testing will need to read another article or two before they can begin to understand what Savoia *maybe* wanted to say. Believe me, these people will not take the time to do that. Heck, these people will not take the time to watch the keynote itself for longer than 5 minutes. They will just get “Test is Dead” and run with it.

So that is why I’m angry at Savoia. He should have known better. This wasn’t the first time either.

Picking up the pieces

However, it’s not only about him.

I take failures of communication such as this as something to learn from. I firmly believe that, as testers, it is our job to exercise critical thinking whenever we are presented with any sort of information. But more importantly, we must be aware of not falling into the context problem trap ourselves.

That is: when talking specifics, we have to make sure it is understood in which contexts our advice might not apply. This is true both on the job and for the wild web.

So is that it? Just constantly keep reminding everyone about the values of context and thinking with their own heads?

Not necessarily… for that first part that is.

An alternative is to make sure that what you say is abstract enough as to apply to almost all possible contexts of software testing. Most heuristics for example can be applied very broadly – and if not, hey, heuristics are fallible by definition, right? The same goes for ideas like test framing, considering the difference between checks and tests, or the practice of working hard to master your craft.

As you probably noticed from the links, James Bach and Michael Bolton often pull this off very nicely in their blogs. However, they still make a point of reminding people about the “best practice” fallacy.

However: Is it actually possible to cover all the different aspects of testing with only broadly generalized terms and ideas?

Is it wise?

More to come.