Kent Becks 3X Model and Quality (a quality coach perspective)

In his article, the product development triathlon, Kent Beck proposes that as a product/company evolves, value-maximising behaviour also changes.

The 3X model he uses identifies three distinct phases in a product lifecycle; Explore, Expand and Extract. In 'Explore', we anticipate low payoff and low customer success while we experiment and 'discover' our value add. But on hitting that flex point and discovering value, we hope to expedite payoff and customer success quickly. 'Explore' is our growth phase. And finally, in the 'extract phase', where customer success is static, value is maintained through efficiencies that provide the desired payoff.

Kent Beck best describes it as follows:

"Explore–the risky search for a viable return on a viable investment. Successful exploration is unpredictable, so the highest expected value strategy is to reduce the cost of experimentation and put a little investment into many, uncorrelated experiments. If you’re lucky, one of these experiments turns out to be unexpectedly successful, which leads to:..

Expand–now things are going nuts (think Pokemon Go or Facebook Live Video). Unanticipated bottlenecks appear. All you have time for is to eliminate the next bottleneck just before it derails you. Once growth becomes routine, it’s time to:

Extract–now the shape of the problem and solution spaces are clear. One euro in equals three euros out. Playbooks emerge: here’s how you roll out the service in a new city. Economies of scale matter: delivering the service at lower cost is more profitable."

Value is measured through payoff and customer success and should follow a growth curve that is not linear but s-shaped. Understanding and having reasonable expectations of value per phase allows us to maximise the behaviour that drives such value.

According to Jerry Weinberg, value and quality go hand in hand ¹. By extension, quality will follow a similar trajectory. If we know our product/company's phase, our quality strategy can adopt appropriate behaviour for that context.

The rest of the article describes possible quality strategies for each phase.

¹ "Quality is value to some person" Gerry Weinberg -Quality Software Management: Systems Thinking

Explore

In the explore phase, the focus is on conducting as many cheap experiments as possible to identify value add. When considering a quality strategy, optimise for the following:

• Customer Feedback
• Low-Cost Strategies
• Speed to market

This approach doesn't mean we ignore software testing. In this context, delivering value rapidly is quality. Therefore, any software testing performed needs to be considered in that context. Startups instinctively do this, delivering quality outcomes without having a software tester or QA on the team. And, of course, achieving quality is more manageable when everyone fits into one room. Here, inherently, you have context and are aware of what is happening and why. You probably wrote the code, and that knowledge allows you to fix and respond faster to incidents. And, your customers, knowing you're in startup mode, might be more tolerant of failure.

Do Quality Strategies in Explore

Quality strategies in the 'explore phase' should be inexpensive and facilitate customer feedback. Invest in habits and rituals that encourage a culture of quality regardless of the product you're building. Think TDD, pair programming, retros, small batches of value, all the good stuff you might think is overkill for the experimental phase.

• Functionality
  Good unit test coverage, then exploratory testing behind a feature flag. Exploratory testing aligns with customer value and is a fast, effective way to find bugs.
• Security
  Ensure engineers build with security in mind (dependencies, code, data and infrastructure).
• Observability
  OpenTelemetry will provide you with observability and the ability to recover from production failures quickly. It's an inexpensive way of building quality into your system.
• SLO's
  If you are going to measure anything, let it be an SLO. Make sure you use SLO-triggered alerts focused on customer value.
• TDD
  TDD develops that vital software testing muscle into software engineers. Just do it; you're future self will thank you when you have to break up that monolith.
• Vertical Slicing
  Get into the habit of teams delivering on vertical slices. When teams expand and grow, these habits will become lifesavers
• Find & Fix
  Get into the habit of fixing bugs as you see them. Avoid testing phases and bug-fixing phases.

Maybe Quality Strategies in Explore

• Performance Testing
  Will the number of subscribers justify the cost of performance testing? If you choose to include performance testing, make it cheap and dispensable.
• Security Testing
  This one is tough. A security leak or a hack can severely impact your reputation. But do you have a reputation? Context plays a massive part here. Consider your product & your clientele. If you choose to perform security testing, I would prefer outsourcing so you can focus on building features.

Avoid Quality Strategies in Explore

Never say never, but I wouldn't be doing the following:

• Detailed documented processes.
  By the time you've finished the documentation of your process, it will likely be outdated. So save this for when you're in the expansion phase.
• Large GUI test automation suites.
  Wait until you know you're value add before investing in GUI end-to-end test automation. Typically there's a high cost to these automated tests. If you must, limit the number, and use the time saved in maintenance on exploratory testing.
• A large number of metrics. My definition of waste is collecting large amounts of data that fail to inform decision-making. Be intentional about what you choose to measure. Drop metrics not in use.

Expand

Congratulations! You've discovered your value-add, and you've pivoted. So let the crazy world of growth begin! In the Expand phase, crazy growth means lots of things that used to work well begin to break, and quality begins to suffer.

For example, due to the crazy hiring that's going on, software engineers no longer have context and are unfamiliar with the code. The person on-call is no longer the person who built the code makes debugging and fixing harder. Not everyone is in 'the meeting', and decisions are lost. And, those trade-offs you made in the explore phase? Yep, they're coming back to bite. Welcome to the world of tech debt and scaling.

In this phase, the only constant is change, and it's happening everywhere. Oh, and you will rarely have sufficient budget to cover all the things ^tm. In the Expand phase, intentionality and agility are your friends. In my quality strategy, I'd optimise for the following :

• Scaling context, both domain and technical
• An emergent quality strategy resulting from experimentation
• Practices, rituals and tools that build quality into the system

(note from me: that was a tough list to distil!)

Do Quality Strategies in Expand

Remember all those practices I encouraged you to optimise for when you were in the explore phase? They're your foundation, and it's time to build on them as everything scales. I've also included them in the quality strategy below. These practices and rituals may be harder to roll out now, but that doesn't mean you shouldn't try. Think of how you can influence people of their value, as the pressure to deliver quickly is in front of people's minds.

• Exploratory Testing
  Exploratory testing is a fast, effective way to find bugs, and there will be plenty of those as your systems struggle to scale effectively. Exploratory testing has a side benefit: it builds domain and technical knowledge. Lean into bug bashes. Consider onboarding practices such as sandboxes with intentional errors for software engineers to find. Much more fun and engaging than reading countless confluence pages.
• Test Automation
  Naturally, you will be creating unit tests, preferably through TDD. I like to also focus on the API layer in two ways. Contract Testing and Functional Testing.
• Security
  Along with building security, it's to be intentional about security testing. Think tools, process, infrastructure and product risk and develop an approach that is lean and agile that works with the product teams. Avoid creating additional bottlenecks in the system by attempting to overlay a top-down approach.
• SLO's
  OpenTelemetry and observability tools such as honeycomb allow you to move from the concept of 'perfect software' to fast recoverability. SLO-triggered alerts that wake you up only when customer value is impacted.
• Accessibility Testing
  Quality in the 'experiment phase' can be pretty one-dimensional. That is, "did it work" and "did the customer like it"? Now you're a growth company; your product needs to work harder. It should cater for a diverse set of customer needs and customer locations. One crucial quality attribute is accessibility. Do Accessibility testing behind a feature flag in production.
• Usability Testing
  Your product was once relatively simple and possibly designed and developed by one team. Now you may have multiple teams working independently on different features. This can become a usability nightmare! Design for usability in mind, get it into those discussions early by encouraging engineering and designers to collaborate and discuss usability early on. Use the concept of local previews to help early identification of design issues.
• TDD
  If TDD is not part of your software development practices, it's worth experimenting with this practice. As a quality practitioner watching software engineers use TDD to build and design systems is pure joy, especially if matched with pair programming.
• Pair Programming
  Pair programming is excellent for ensuring culture, practices, skills and ways of working are embedded into new hires, consider pair programming. Most people learn from doing, not reading confluence pages on high-level principles. There is so much benefit to discussing how to test something with another person.
• Vertical Slicing
  Like TDD, one of these experiments will be a win at some point. Get into the habit of teams delivering on vertical slices. When teams expand and grow, these habits will become lifesavers

Maybe Quality Strategies in Expand

• Hiring Software Testers
  The expand phase might be an excellent time to consider hiring a software tester. Remember that quality is a team responsibility. Adopt a whole team testing approach, where the team takes on software testing as a collective responsibility. Sometimes, a software tester's work can become a bottleneck. You can avoid 'testing bottlenecks' by having software testers involved in all team conversations, allowing them to test early and often, and having social contracts where the team agrees to contribute to any testing work.
• Performance Testing
  Scaling should mean that the number of customers is going through the roof. You're systems and architecture designed for startup mode probably can't cope. So, you beef up your cloud storage, memory, and CPU to keep the lights on; consequently, your costs start to go through the roof.
  Performance Testing can provide visibility of where systems begin to suffer, which is valuable knowledge. But Performance Testing can be expensive, so consider where it will add value. Monitoring latency in both development and production maybe be a more effective strategy.
• Enablement Teams
  Team Topologies is your friend. Consider a quality engineering enablement team that works to offer support to product teams in terms of developing test automation templates within repos. Look to embed testability and observability into architecture blueprints.

Avoid Quality Strategies in Expand

In Explore phase, the quality strategy needs to be agile and scale rapidly, so avoid the same approaches.

• Define a detailed quality process across all teams
  In my experience, these processes will give you visible structure, but unless everyone's agreed to follow the process, it will probably lie, ignored in a confluence page somewhere. Instead, focus on localised solutions within teams and camps. It's tempting to look for alignment to provide consistency and repeatability in the hope of some efficiency. Do you want to optimise for efficiency at this point, though?
• Large GUI test automation suites.
  Now you have decided on customer value; surely it's time to put those Playwright tests in, yes? But hold on, not just yet. You still want your software testing to be agile and respond to changes in scale and direction. Lots of test automation is possible, but it comes at a cost in terms of maintenance and slowing you down. So instead, focus on the API level. Think of Pactflow, which offers both contract and functional testing.
• Hiring a junior QA to 'fix' quality
  Think strategically about your quality strategy. Expecting them to solve all your quality problems is unfair and unrealistic. Quality is a team responsibility, and sure a tester will help find those bugs, but developers still have to fix them. If you're hiring a junior QA to remove bottlenecks, then I'm sorry to be the one to burst your bubble. All your doing is shifting the problem elsewhere, and soon your QA will become the bottleneck. Alternatives might be to hire a QA per team or consider quality coaching.

Extract

The extract phase of product evolution suggests either market saturation or reaching as far as you can go regarding customer success². You've demonstrated customer value, and consequently, the rate of change to customer value decreases. The extract phase is where you look for efficiencies to increase business payoff.

In your quality strategy, you are optimising for:

• Tooling & Framework consistency
• Waste identification and reduction
• Compliance & Reputation concerns (SOC2 & audits)

Do Quality Strategy for Extract Phase

Efficiency is the word of the day. The 'expand phase' purpose was to scale; in the 'extract phase', the fundamental goal is to identify efficiencies in both technical and internal business processes. You will continue to pay off technical debt and lower risk through technology updates.

In the Extract phase, our systems may be the most stable they've ever been. But there is still a risk of failure. While the likelihood of failure is reduced due to fewer features, a large customer base results in a more significant impact. Therefore, in the 'Expand Phase', it makes sense to focus on increased test automation.

Our focus is on delivering efficiency through speed and reliability. For example, some strategies to consider are:

• Automation
  Identifying manual steps that are repeatable allows the opportunity for greater automation. Obvious optimisations exist in regression testing and manual runbooks, but also consider automating internal business processes that improve efficiencies.
• Playbooks and Ways of Working
  Now is the time to embrace ways to keep people informed and aligned on practices, rituals and ways of working. Confluence pages, in the past, may have been localised to a team or a squad. Departments can begin to identify efficiency by creating playbooks and practices aligned across the whole department. And, now that the rate of change has reduced, these artefacts are less likely to become outdated rapidly.
• Metrics
  Now that change has reduced, outcomes have become more predictable, and it makes sense to have leading and lagging metrics. DORA Metrics, in particular, are helpful as they track unplanned waste
• Security
  Your focus is on repeatable and consistent processes that external companies will regularly audit to ensure compliance. Audits and compliance have probably already been introduced in the growth phase, as it is part of a scaling strategy.
• Enablement Teams
  If you haven't done so, think about creating an enablement team focused on developing quality-related blueprints for product teams. Product teams should be able to pick up these templates and blueprints with minimal effort.
• Monitoring and Alerts
  Pretty much indispensable in modern engineering architecture. I almost left it out because it seems as apparent as unit testing.

² it's rare in SAAS companies that we remain in the extract phase for too long. Instead, they circle back to explore new features and new products.

Avoid Quality Strategy for Extract Phase

There's not much to avoid when you're in the 'extract phase' because these systems can absorb a lot of poor-quality practices. So much is outside your control, and you will have limited opportunity to drive significant change other than something that increases efficiency.

Avoid quality strategies that don't align with delivery efficiency. Instead, work to build relationships across departments to ensure that product quality doesn't drop.

Where does Quality Coaching fit in?

In the explore phase, I would encourage a senior software engineer to take on the mantle of a quality coach. This person thinks strategically about the trade-offs required to deliver a quality product. They would also think about where and when is the optimal time to test.

In the 'expand phase', I'd consider how experienced teams are in software testing and quality-related activities such as SLOs and monitoring. If a significant effort is required to improve quality, I look to put a software tester on the team to coach the team on whole-team testing. Then, when the team reaches a level of maturity, I'd move to a quality coach model. You could, of course, have a quality coach model straight away, but it's a big ask for the team to step to that level, so be prepared for a long journey of quality improvements.

A quality coach model makes perfect sense for the extract phase of a product lifecycle. It delivers the efficiencies required for that stage. In addition, it's well suited for identifying gaps in quality where waste exists.

I feel like I've only scratched the surface of this blog post. There is a whole heap of other elements to consider. For example, I haven't even touched on where and when I would focus on data quality or where AI fits into the picture.

And, of course, there's your product context to consider. I've predominantly considered a Saas product in a high-tech company. Your context will shape and influence your strategy in ways that may make the above suggestions counterproductive.

What would you do differently? Please share!

Ask the Book

Welcome! Ask anything from the Handbook.

How can I start improving quality without slowing delivery? What should I focus on first as an engineering leader? How do I coach teams toward better systems thinking?