Quality Solution Opportunity Tree

Quality is an ecosystem

The domination of SaaS as a product model and the drive to deliver software faster has dramatically impacted how we think about quality. Testing phases no longer exist. Instead, they're replaced by "testing moments". Where in the past we looked to software testing as the primary quality approach, we now look to build quality into our systems, using various tools, processes and practices that combined to allow quality to emerge.

Our quality strategies look more like this:

Silver Bullets Ahoy!

That doesn't stop the obsession with solving quality with silver bullets. Each boss has their pet silver bullet. I have my pet silver bullet. As humans, we can't help ourselves anchoring onto something we know will fix the world. Bless our little cotton socks!

In the quality space, the terrain is littered with silver bullets of various sizes and shapes. And, as we grow more experienced, we recognise these as over simplistic solutions to difficult and complex problems. As best we can, we continue to educate our colleagues on what 'good looks like' for a quality strategy. It's a Sisyphean task.

Not all silver bullets are equal

I like to pay attention when my CTO and senior leadership talk about silver bullets. Typically the conversation begins by highlighting either one of the following concerns:

• Testing takes too long
• There are too many bugs in production
• The feedback from testing is too slow
• We can't see that state of quality
• Testing is too expensive

Their favourite silver bullet rapidly follows.

Over the years, I've made a list of them:

• We need more test automation to speed up testing
• We need increased regression testing to reduce bugs
• We need increased unit testing to provide faster feedback
• We need better testing metrics will help see the state of quality
• We need to outsource to reduce the cost of testing

Here's a visual of the problem and solution space.

Silver bullets come with a price tag

We know these solutions are alloys made of some cheap metal rather than being made of silver. Often they go some way to solving some of the quality issues out there. But, more often than not, they become more expensive, complex and harder to roll out. Who knew software development was so hard?

Example: test automation decreases our test automation from five days to one. We also need to hire double the number of test automation engineers, who all need suitable lead times to create these automated tests.

Still, that doesn't mean that the solution is inherently wrong. It's more that we know various solutions are required to improve quality.

Diversify your strategy

A solid quality strategy relies on more than a silver bullet; instead, needing a variety of tools and tactics in your toolkit. Or, as I prefer to say, you need more than red lipstick in your makeup bag.

Consider a range of tactics and approaches to improving quality. As the above map suggests, a lack of quality is rarely down to one thing. It's a range of problems from understanding the customer, and the complexity of the solution, to how internal teams communicate and collaborate and customer support.

Teasing folks away from their silver bullet is not easy. I advise avoiding taking the high road by dismissing these ideas outright. Buying into some of your boss's 'silver bullet'¹ solutions is not altogether a stupid strategy. It will open purse strings, give you continued support (important when things get tough) and help you frame reporting to senior leadership.

Instead, I ask, "How else might we improve quality"?

The "How else" quality framework²

This framework asks at a high level: "How else might we improve quality?". It's a simple open-ended question that encourages alternative perspectives, a good antidote to tunnel vision and SBS (silver bullet syndrome). Here's an example of the framework

I like to bucket solutions into three categories, prevent, detect and recover. Prevent, detect, recover is a powerful heuristic that allows you to frame how quality needs to be at the start, the middle and the end of our product services.

• Prevent: What solutions allow us to build quality in?
• Detect: What solutions allow us to detect quality-related issues?
• Recover: What solutions allow us to recover from quality-related issues?

By focusing on prevention, we theoretically can reduce the time required for testing, reduce incidents, and reduce the cost of testing. So I like to focus most effort on the prevention part. Your company may need a different focus.

You can prime this conversation by placing some solutions into each category and encouraging others to suggest their ideas.

Once the brainstorming is exhausted (anything between 3 and 20 minutes, depending on the group), it's time to prioritise what solutions you as an organisation want to invest in. Using a DACI is a popular model that allows the group to explore the pros and cons of each solution. I like to consider the impact on quality, short-term gain versus long-term gain, and cost (in both time, people and resources).

Workshop Material

In the Miro board³ below, I've linked the ideas to the original problems using the prevent, detect, recover heuristic. This connects the original problem to solutions.

In reality, many of these solutions solve multiple problems. Or they create new problems. So it will never be clear cut. Still, it goes some way to moving the conversation forward in the right direction.

³ Miro Board of a Quality Opportunity Solution Tree

You are welcome to make a copy of this board and use it for personal purposes.

Also, I've put a pdf of the images in this post as a download.

Lessons learned

Some things I've learned over the years about defining strategies and solutions:

• Offer pros and cons of options so people can understand what the choices they are making
• Frame the execution of ideas as experiments that can be retired if unsuccessful.
• Solutions don't have to be perfect, but they should be realistic to the context of your organisation.
• Be mindful of your own bias. The answer might not be the most perfect, but it may be the most practical.
• Give people early wins. Education comes in time. It's a marathon, not a sprint.
• Nothing is forever; if you are overridden on an option, work to provide a date by which you can review the success of that option.

Finally, don't worry if people return to choose their favoured silver bullet. Changing hearts and minds takes time. It may take lots of similar conversations before it becomes safe for people to invest in new ideas.

Footnotes

¹ Of course, if it is idiotic and dangerous, you should be vocal about that. But try to find ideas or approaches you can agree on.
However, sometimes neither agreeing nor disagreeing with an idea and quietly let it die peacefully alongside other stupid ideas in the wasteland of rampant stupidity.

² This concept is built on Teresa Torres Opportunity Solution Tree

Quality Coach Assistant

Welcome! Ask about quality coaching, leadership, and engineering culture.

How can I start improving quality without slowing delivery? What should I focus on first as an engineering leader? How do I coach teams toward better systems thinking?