Devs & testers will always debate about whether a bug is really a bug, and such discussions are healthy, up to a point. If these debates happen often, they take time, energy, and extremely valuable focus, from building or changing features, improving design, evaluating risk in other scenarios, and pressing needs.

When we incentivize testers to find bugs, they will find bugs, or at least things they can label as bugs. This will eventually lead too many debates.

As with almost everything in software development, more is not better. Finding more bugs will not lead to more productivity, and possibly not even quality. For testers, the focus on quantity will have them spending most of their time searching for bugs where they believe they can find plenty (of possibly trivial bugs), instead of stepping back regularly and evaluating the overall risk of the system.

Focus on Risk Analysis

At O&B, we call our testers “Test Analysts“, to emphasize the analytical nature of their work (I wanted to call them “Risk Analysts”, but that term was taken). Their primary job is to understand and analyze the whole system, vis-a-vis business priorities, and map out the heat map of risk. E.g…

• “How much money can the company lose if this scenario happens?”
• “Can we get sued for this scenario?”
• “Can we permanently lose customers in this scenario?”

Testers, or test analysts, are not much different from business analysts, but their focus is what can go wrong. They are the voice of the scenarios of risk.

Risk Advocacy

After finding risk hot spots, the job of the tester is to engage both engineers and product management on how to best mitigate the risk. Engineers can come up with automated tests. Product management might change the feature specification to side-step the risk, or might even decide the defer the feature if they assess that the benefit is not yet worth the risk.

Holistic Incentives

Incentivizing testers to simply find bugs will incentivize them towards a very narrow focus, when their roles is much broader and more valuable than that. As with all things in Agile, incentivize the team on holistic business outcomes, not on very specific outputs.