Experiments in Quality Coaching
Use the concept of experiments to encourage product teams to try new approaches, tools and ways of working
Experiments are powerful methods for driving change within product teams. An experiment provides the team with optionality, offering psychological safety. They can try something out, opt for the new change, or revert to working as before. Experiments permit teams to try new things without committing long-term to change. They offer change without removing team agency; at all times, the team decides.
What is an experiment?
An experiment is a procedure or operation carried out to resolve an uncertainty1. For an experiment, you form a hypothesis upon which experiments are designed and then run to see if that hypothesis is true.
As a quality coach, you can suggest to a team that they try a new idea or concept as an experiment. It is even better if the team comes to you with a suggestion. The team can then try the idea; if it works, they can absorb it into their working methods.
Learning opportunities
Experiments can pass or fail. Failure is data that suggests the hypothesis is untrue, that assumptions are incorrect or that there are unknowns to uncover. The failure of an experiment is not a failure of the team, the quality coach, or even the proposal put forward.
Viewing experiments as learning opportunities, rather than an admission of fault, encourages a team to keep evolving and experimenting within the problem space.
Failure offers the opportunity to better understand the problem at hand and then make informed decisions about how to modify your assumptions about the problem and the solution, modify your approach and perhaps perform another experiment.
Dealing with Time Constraints
Teams manage and absorb much change as part of their daily work. Keeping a local machine 'clean' requires constant updating and upgrading. Engineering teams do a significant amount of housekeeping. They maintain CICD pipelines, libraries, and frameworks and sometimes test data and test environments. A software engineer once told me that a good day's work is if you can write code for three hours!
With this amount of change, it's understandable for a team to be reluctant to try something new. This is especially true if a concept or tool doesn't appear to solve a team problem. But maybe a small experiment might be achievable. This is the beauty of experimentation. With a small investment, a team can opt out of long-term implementation if the value is nonexistent.
Scaling Experimentation
One concern I often hear from senior leadership is that the experimentation approach doesn't scale when you want to drive change across large enterprise organisations, which requires consistency of approach and adoption.
Rather than focusing on process, I encourage focusing on the behaviours of continuous learning and problem-solving, as we know these traits are useful for organisations to respond effectively to market demand.
Encouraging an experimentation mindset is one of many levers a company can use to adopt this mentality.
If you are interested in driving systematic change using experimentation, read Neil Younger's articles on building an engineering framework around capability and growth.
Neil Younger
Neil Younger
Case Study on Experimentation
A team once approached me to experiment to trial Behaviour Driven Development (BDD). They hypothesised that this would provide certainty at the user story level. They already used TDD and felt that BDD would take their automation to the next level. like the concept of BDD, but teams often ignore the value-added bits, such as example mapping and jump straight into test automation. I was reluctant, but the team was enthusiastic and wanted to try it out. Since being responsible for quality means teams have agency over decisions on quality, the team opted to perform it as an experiment.
They decided to try out BDD for three months. They held workshops and analysed the stories, breaking them into the Given, When, Then format, making it easy to use Gherkin and Cucumber scripts to automate the behaviours.
Three months rolled by, and it was time to make a call. The team almost unanimously agreed that the approach was a lot of additional work for little value added. Their context was financial back-end transactions, and they found it hard to apply the user-centric approach that BDD encouraged.
They opted to drop BDD and continue using a mix of Test-Driven Development (TDD) and Contract Testing, which offered sufficient test coverage.
Not everyone agreed. The one person who wanted to keep BDD was the software tester!
Considerations
As a method, experiments are powerful ways to encourage change, but be careful how and when you use them. Here are some guidelines:
- The team must have agency when and if to implement an experiment.
- The experiment should solve a problem that matters to them rather than what you, as a quality coach, think the team needs to solve.
- Frame any experiment as an opportunity to learn and understand something new about the context. What did the team learn if the problem still exists, and what might they try next?
- Keep experiments small and simple.
- Avoid making decisions about the next steps until the experiment has been completed.
- Be clear about how success will be measured and what it might look like, whether that's quantitative, qualitative, or both.
- Teams differ in their openness to experiment; if you are trying to drive systematic change, try to find a team that a) has the time and b) is open to trying new things.
What will you try as an experiment?
Comments ()