Team Topology & Quality Engineering

As director of quality engineering, part of my role is structuring how and where quality sits within a company that's scaling in customer numbers, in terms of architecture and software engineers. Team Topologies written by Matthew Skelton & Manual Pais suggests reversing Conway’s law and providing structures that facilitate collaboration (with the proviso that not all collaboration is good).

In this post I'm exploring how this might apply to a quality engineering practice.

Team Topology & Quality Engineering

Almost instantly it feels like the QE practice is an enabling team. One that supports Stream Aligned (SA) teams achieve their goals. Makes sense, especially in terms of test automation and a quality assistance model. But as I dug deeper in the concepts, I realised it was a bit more complicated than that. Yes, test automation fits into that paradigm, but what about exploratory testing? Where should that sit, and how should it be supported?

To be able to answer these, I had to look at what is meant by quality, flow of work and cognitive load theory (in particular germane load).

Quality is Throughput

Quality relies equally on how stable our value is, & how fast we get that value into the hands of our customers.

Quality is how fast we deliver value (throughput), how stable that value is (stability) is & our ability to deliver on our promise of quality. – Anne-Marie Charrett

Our ability to deliver value rapidly relies on small batches, staying in the flow of work, minimising rework, & automating what makes sense. These approaches have the added benefit of lowering the risk of any change, also stabilising value.

This means we can't exclude how Quality Engineering activities impact our ability to deliver at speed. It's not just about how well we test, it’s also about how well we deliver. Quality is an ecosystem and everything must be considered if we really want to make an impact.

Software Testing is Germane Load

Matthew Skeleton's position is that any software testing that benefits from significant domain knowledge should stay in a Stream Aligned Team.

Software Testing is predominantly germane load. That is, it's the type of cognitive work required to develop mental models of our systems. This sits well with my understanding of software testing, and so I agree that we can predominantly place software testing into Stream Aligned Teams.

Reducing Cognitive Load

But here is where things start to get tricky. To maintain a flow of work, we need to reduce any cognitive load that distracts from this work. While we can argue that creating & running software testing is part of that flow, there's no avoiding the fact that software testing brings extraneous load with it. Think test environments, test data and test frameworks. When all you want to do is run a test, you end up in hours of yak shaving.

To avoid this, we can create a QE Enabling team that focuses on these activities. Many companies are doing this already, so little controversy here.

Exploratory Testing is about being distracted

This all works very nicely when we think about automating our tests, but what about Exploratory Testing?

Exploratory Testing moves outside the flow of work. It requires focus on that which is distracting. Performing Exploratory Testing is to indulge curiosity, to jump outside stories, to pull on threads of information that flow right across and down into our systems, regardless of bounded context or team scope. It doesn't march to the drumbeat of delivery, instead preferring the unspoken, the unsaid, and the unseen.

The impact of exploratory testing on the flow of work is disruptive. It often uncovers rework or unplanned work, and requires context switching. These are exactly the types of activities that slow us down and upset the flow of work.

But if we don't perform Exploratory Testing early on, we run the risk of further disruption down the road, longer feedback loops, greater context switching, more handoffs.

Here are three approaches to consider. I've created diagrams of each in a link at the bottom of the post

Exploratory Testing into the engineer's flow of work

One approach that's very popular is to place all software testing in an engineer's flow of work. This means an engineer is responsible for both the development and testing of a feature, including both test automation and exploratory testing.

To succeed with this approach, software engineers are trained to perform exploratory testing. Touch points within the flow of work act as guardrails maintaining the quality of the testing effort. And to minimise bias, software engineers can test each other's code.

The benefit of this approach is that it empowers the software engineer to design for and include testability in their solutions. Also, as they begin to build a mental model of typical bugs, they can begin preventing these bugs from appearing in the first place. There's no safety net, no tester waiting to catch bugs in case they're missed, encouraging engineers to think twice before building and shipping code. It takes time for this knowledge to be acquired but it can be worth it. Biggest plus is less handoffs, less rework.

Exploratory Testing in a team’s flow of value

By including an exploratory tester in a team and focusing on the flow of value, you acknowledge that a certain number of handoffs will exist, but that the value gained justifies this approach.

Plus you get an added bonus.Teams begin to collectively absorb knowledge about risk to the point where they build software with those risks in mind. In this way, the role of the exploratory tester almost becomes redundant. Handoffs, rework and context switching becomes minimal. It does take time for a team to reach this point, though.

I've seen this work amazingly well, but I've also seen it create tension and unhappiness in teams. It takes a concerted effort by the whole team to be open to new ways of working, trying and retiring if things don't work, plus a healthy respect for all disciplines.

Exploratory out of the flow of work

There is a third concept which I've coined the Eclair Model.

Instead of injecting Exploratory Testing into the flow of work, the focus is on performing exploratory work pre- and post-delivery. Exploratory Testing in production consists of monitoring and analyzing data as your customers use the system. Observability is critical here, with latency a key measure for performing root cause analysis. The motivation is to better understand the system and, in doing so, better understand risks.

It is critical to bring this information back into ideation and discovery, helping product owners and product analysts understand the system as it behaves, using their mental models of risk to identify potential concerns. Pre and post delivery there is room for coaching on system risks.

To summarise, the approach could be: Build the Right Product >> Feeding Risk & Data back to the product team Build the Product Right ->> software engineers with an enabling team Support the Product Right >> Exploratory Testers >> Monitoring & Analysing in Production

What's right for us?

There are a lot of variables that will influence your final choice and approach, including existing cognitive overload in your teams, the level of willingness to learn new skills, and the capabilities of those doing exploratory testing.

Speak to the SA teams and ask them what concerns them most, silos or cognitive overload? What is the appetite for observability in production? How open are your product teams to include someone with a testing background? All these will influence what approach works best for you.

Here are some other team topology QE structures to consider Team Topology diagrams of QE structures

Many thanks to Fiona Charles for reviewing this post. And thanks to everyone who has signed up to this book!

Background Reading

Key concepts are described on the Team Topology website. I found this book summary from Dan Lebrero useful

Jesper Ottosen has already written on how "testing people" might fit into the Team Topology structure. which is also a good read and offers some good insights.

Cognitive Load Theory (CLT) and the concepts of intrinsic, extraneous and germane load.