Improving Agile training at Cancer Research UK — part 1
When I arrived at Cancer Research UK, the Agile team were preparing to use Objectives and Key Results (OKR) for the first time.
One objective the team was keen to work on was improving their agile training offering. I spent some time with the Agile Coaches to gauge the motives behind this and to understand what they needed out of training. I also had the opportunity to shadow my colleagues delivering one training session — Agile Fundamentals.
Two weeks in, as a team, we decided that we would try out the goal-setting methodology with two OKRs. I asked to take ownership over the second one: Improve basic Agile training modules. Taking ownership of this OKR meant that I would be responsible for reporting the progress that the team were making towards the objective.
At that point, I was the person with the most bandwidth to take on some work. I decided to dedicate a week to work on some initial discovery and involve my colleagues at key points in time.
Each of the three Key Results was broken down into a first iteration of initiatives (individual steps that describe what you will do to achieve your objective), which later evolved as I gained more context and knowledge on the subject.
Providing context
The first activities that I conducted were around unpacking existing knowledge of the agile training offering. This included identifying the main business drivers behind agile training at Cancer Research UK. I identified business goals by consulting different strategies in the Technology directorate, which included embracing new ways of working, enabling collaboration, and embedding continuous improvement and a test and learn approach.
During this context-setting phase, I also sought to map out how people seek and receive agile training at Cancer Research UK. This resulted in a journey map which identified other areas for improvement (we’ll keep this for later!). I also identified the type of people who sought or could seek training in order to interrogate whether the training is or can be designed for them.
I then reviewed the training material of two different modules — Agile Fundamentals and Agile Methodologies in order to understand the work that had already been done to review the training modules and their learning objectives. Once reviewed, I identified the current and past training learning objectives. As part of this review, I conducted a SWOT analysis of the training modules with the Agile Coaches in order to identify the strengths, weaknesses, opportunities and threats from the perspective of those who deliver the training. This was carried out with the intention of picking up what might not come out of the participant’s responses to the pre and post training questionnaires.
At this point, some questions started to arise. Are these the right learning objectives? Is this what people want or expect to learn? Does the training content meet the learning objectives? Is training even the right format to help the business achieve its goals? Motivated by these questions, I moved on to defining the problem and drafting an initial hypothesis.
Defining and fleshing out the problem
Seeking to understand what people want or expect to learn, I looked into the pre-training questionnaire that is sent to all participants before the session. The questionnaire consists of questions that gauge the participant’s knowledge on agile and their expectations of the training — the latter providing qualitative data that I was then able to analyse and group participants’ expectations and needs into themes:
- Improving ways of working
- Broaden agile knowledge
- Relevance to Cancer Research UK
- Personal development
I then asked the Agile Coaches to dot vote the five participants’ expectations they believed would bring the most value if we were able to determine whether we could meet them through our training offer. The expectations below will inform the development of a set of training learning objectives that will be tested with participants and will shape the content audit at a later stage:
- Learn practical tips to improve ways of working
- Know and be able to explain what agile is
- Know how to encourage an agile mindset in a team
- Understand what agile means for CRUK
- Understand how agile is applied in a non-tech environment
The post-training questionnaire also offered insightful data on whether the training met the participants’ expectations and what could have been done better. One of the five quantitative questions particularly stood out: “How confident do you now feel to use the learning from this session in your role?”. In both training modules, this was the question that scored lowest. We believe that this deserves more attention as it defines the core foundation of the problem space: people who participate in the basic agile training sessions are not as confident as they could be in applying what they learn in the agile training sessions to their role because they feel as if they leave without any tools or techniques to do so.
As an assumption, we believe that if we could solve this problem, it would impact the participants positively by providing them with just enough knowledge to start applying what they have learnt to their day-to-day role. This would also satisfy the business’ appetite as it would promote a way of working that is more collaborative and focused on continuous improvement.
At this point, I started to think about ways that this could be validated and evaluated — which I will address later on in this post. As an initial hypothesis, we believe that if we can better manage expectations through clear learning objectives (and refine the training content to better reflect them) and demonstrate how agile values, principles and practices can be applied to participants’ context of work, participants will feel more confident in applying their learnings to their role.
All the while, throughout these two phases of discovery we were continuously acknowledging, challenging, testing and sometimes validating our assumptions which proved to be a useful tool when it came down to defining a hypothesis as we were able to pick a focal point for experimentation and maximise learning based on the five participant’s expectations we decided to focus on for our first iteration.
Exploration
Having defined the problem space, this exploration phase sought to look further into the themes and patterns that have come up throughout the previous discovery phase. We also wanted to understand what “good” training looked like by exploring external training offered by agile trainers and certification and accreditation bodies such as ICAgile.
We then dove deeper into the post-training questionnaire to understand what participants most found useful and what they thought had room for improvement. By analysing the participant’s responses, we identified that they found the training to be useful and engaging — especially when using activities to explain concepts. However, many participants felt that there was no attempt of contextualising agile to the work that takes place at Cancer Research UK, and that they left the training without understanding how they could put their learning into practice, as noted by a participant’s feedback:
“I think the training could be excellent but as it was all new methodology and there were no relevant examples of how we could apply it to our type of work it felt difficult to see the obvious application.” (anon., post-training questionnaire)
This suggests that the training, in part, doesn’t meet the participant’s initial expectations as laid out in the pre-training questionnaire.
From this analysis, with the prioritised list of participants’ expectations, both the positive and negative statements were transformed into opportunities by framing them as How Might We (HMW) questions:
- HMW concisely explain the agile principles in a way that is relateable and relevant to all types of participants?
- HMW create a balance between communicating agile as a mindset and providing some practical tools and techniques to put agile into practice?
- HMW include examples of agility in Cancer Research UK in a way that non-tech people can relate to?
- HMW improve group activities in a way that participants understand how the underlying message links back to agile?
- HMW encourage participants to leave with an action to apply/explore what they have learnt?
These are the prioritised, overarching How Might We questions that can be used for an initial round of testing. Once learning objectives have been drafted, and the time comes to suggest a first iteration of improvements to the training content, the full list of HMWs will be revisited to see if any others are worth exploring again.
Next Steps
The next steps will involve iterating on the training modules’ learning objectives to reflect the participants’ expectations and what the trainers can realistically offer in a two-hour session. To test whether these expectations can be met, we will need to audit the training content and suggest and implement improvements if there is a mismatch between the content and the learning objectives.
During the exploratory phase, I attended a Lean Workshop session held by Cancer Research’s Service Design team. I bought my work forward, and a small group worked alongside me to think about ways I could get this research out to the organisation to validate and test the assumptions we had made.
We focused on the participant’s ability to apply their learnings to their role. This involves different factors that will need further unpacking. The first involves understanding what level of practical takeaways is useful for participants and realistic for the trainers. The other involves understanding how to evaluate the confidence levels of those applying what they’ve learnt during training — whether it be understanding the different levels of engagement or defining what characteristics participants may display if they feel confident in applying learnings. More to come on this!
As an initial test, we would like to better understand the first factor: how many tools and techniques are enough to enable participants to work in a more agile manner? A possible technique to test this is a concierge test. The idea behind the technique is that you do the user’s job for them so that you can develop a better understanding of them and their interactions. This form of testing shouldn’t be used to validate a solution hypothesis (as it isn’t good for testing out solutions), instead, it can be used to generate new ideas. In this case, this would involve following up with participants to understand if and how they are applying what they learnt, and then providing agile coaching to get them to a point where they feel comfortable in doing so. The gaps in knowledge and/or opportunities identified can then be used to improve the training. This idea will be explored in further posts — watch this space!
Read Part 2 here.
