The testing and revision stage

The preceding analysis has highlighted the importance of the context into which an intervention is introduced in determining its success, and thus the need for adopters to be aware of relevant contextual factors when attempting to replicate the intervention.

But it can be hard for innovators to ‘see’ their own context. When something has been achieved successfully in one location it might seem straightforward to document the actions involved, but it may in fact be impossible to know which aspects of context were relevant to the initial success without being able to compare this experience against other counterfactual scenarios. It may therefore be only when an intervention is implemented in new contexts that the comparative information becomes available to enable the innovator to understand what actually made their intervention work first time around.

This is something we often see in Health Foundation programmes. When an intervention that has succeeded in one setting (or group of similar settings) is introduced into a diverse range of sites, the variable fortunes of the intervention can shed light on which intervention components and contextual factors are more or less important for the intervention’s success. Similarly, as adopters shape the intervention to fit their own setting, valuable learning is created about the tolerance of different components to alteration. This gives the innovator fresh insights into what is and isn’t significant for making the intervention work, enabling them to revise the intervention description accordingly. An example of this process, taken from one of the Health Foundation’s improvement programmes, is described in the case study of RIPPLE at the end of this chapter.

What is happening during this initial ‘spread’ phase – a learning process in which the intervention may undergo substantial re-conceptualisation and refinement – often looks quite different from attempts to spread an intervention at later stages of maturity, when it is codified with greater confidence and issues of fidelity are more clear-cut.

Within pharmaceutical and product innovation, such a stage of comparative testing is usually recognised as a formal part of the ‘innovation cycle’ (for example, the ‘field testing’ or ‘beta testing’ stages of product development), but we have found this to be less consistently so with quality improvement and process innovation. Indeed, we often see innovators applying to the Health Foundation with similar proposals and expectations for spread, despite their innovations being at very different stages of maturity.

There may, therefore, be value in recognising this testing and revision phase as a formal part of the innovation cycle, distinct from attempts to spread the intervention at later stages of maturity, as illustrated in Figure 7. An example of this can sometimes be seen in franchising operations, where the early franchisees make a significant contribution to improving the model, identifying and solving problems, and updating the manual; in some cases, franchisors reduce entry fees for initial franchisees because they know the idea is still under development.

Figure 7: Innovation cycle

This concept of a testing and revision stage is supported by our survey of innovators from Health Foundation programmes. They were clearly aware of the possibility of learning from adopters: 91% said they had learned new things about their intervention from adopters’ experiences of implementing it and, of these, 90% said they had changed the way that they describe and communicate the intervention as a result.

Figure 8: Survey of innovators – learning from adopters

Implications for the design of the initial spread phase

Recognising this initial spread phase as a distinct stage of the innovation cycle, one that is primarily about testing and revising the intervention, would enable it to be designed appropriately. There are several considerations here.

First, it would help set realistic expectations for the outcomes of this initial phase, recognising that the main objective is to learn from variations in performance across different sites, rather than assuming that replication will be successful everywhere. This would help avoid artificial pressure to contrive ‘wins’ from every site and instead focus on the learning generated by both positive and negative experiences. Explicitly acknowledging the need to test the intervention in new contexts could also inform the selection of initial adopter sites to ensure appropriate diversity, rather than creating a bias to work only with sites similar to the original (which might be assumed to have greater potential for successful replication).

Innovators’ openness to learning during the initial spread phase emerged in another of our survey questions. When asked to choose between contrasting statements, two thirds (67%) favoured the statement ‘It is better to select as initial adopters a diversity of sites to test where the intervention will and will not work’, with only a third (33%) instead choosing ‘It is better to select as initial adopters those sites where the intervention is most likely to succeed’.

Figure 9: Survey of innovators – initial site selection

Second, recognising the role of early adopters in generating new knowledge about the intervention has implications for the relationship between the innovator and adopters during this testing phase, where they are cast more as peers engaged in reciprocal learning rather than in the kind of ‘teacher-pupil’ relationship that traditional dissemination models imply. Indeed, we have occasionally seen examples at the Health Foundation where an early adopter makes a greater success of the intervention than the innovator’s original attempt, and the innovator learns a great deal from them. This can sometimes pose a challenge to innovators, who may view themselves as having exclusive knowledge of their intervention, or be attached to aspects of its original form that prove to be superfluous or sub-optimal in new contexts. Innovators therefore need to enter into this phase prepared to revise their own conception of the intervention and to accept that their initial idea will be developed by a wider community. In some cases, where this proves a challenge, innovators may need to bring in others with the detachment and mindset to lead this initial spread phase.

Third, the fact that early adopters are generating new learning that can be used to refine the intervention necessitates mechanisms to capture and share this learning, such as workshops, peer networks and formal evaluation. Here, spread programmes can learn from wider quality improvement initiatives such as improvement collaboratives and clinical communities, many of which invest substantially in creating peer communities to support reciprocal learning. A good example was the Improving Lung Cancer Outcomes Project from the Health Foundation’s Closing the Gap through Clinical Communities programme (2009–2012). Led by the Royal College of Physicians and involving 30 multidisciplinary teams, the project drove a range of improvements in lung cancer treatment, including better access to clinical nurse specialists and shorter referral-to-diagnosis times, through creating a community committed to peer learning. The independent evaluation found that central to its success was a series of peer-to-peer review visits, which enabled sites to learn from each other in a genuinely reciprocal way.,

Such networks don’t necessarily have to be created anew for each new spread programme, though. Box 8 describes one network, the Q community, and the growing role it is playing in the UK in supporting spread and achieving improvement at scale.

In order to better understand whether – and how – interventions work, evaluation is another key part of the picture. Box 9 highlights the work that The Healthcare Improvement Studies Institute is doing in this area to create an evidence base that supports replicable and scalable improvements to health care.

While different structures and mechanisms for sharing learning will be appropriate in different circumstances, the key point is to recognise that the early spread process is often a process of co-innovation – the collective development of an idea through iterative waves of implementation and refinement.

Adopters’ awareness of the learning they generate, and the value they attach to sharing implementation experiences with one another, was reflected in our survey. Nearly all of the adopters who took part in our survey (98%) said that their experiences of implementing the intervention had generated new learning about it. Separately, we asked them whether sharing implementation experiences and learning with other adopters had been a major part of the spread programme and, if so, whether this had been important in helping them to implement the intervention. Of the 91% of adopters who said that sharing learning had been a major part of the programme, a clear majority (58%) said this had been important in helping them (see Figure 10).

Figure 10: Survey of adopters – learning from other adopters

This finding was mirrored in adopters’ responses to a further question, discussed in Chapter 5, about what would have made the biggest difference for supporting adoption; the most popular choice was having more opportunities to share learning with one another.

These survey results speak to the value of nurturing ‘horizontal’ networks of adopters for supporting spread, which can act as a valuable mechanism for sharing learning as well as creating shared vision and values.

Box 8: The Q community

Q is a community connecting people with improvement expertise across the UK. Funded by the Health Foundation and NHS Improvement, Q aims to support people in their improvement work by creating opportunities for them to come together and share their knowledge and ideas.

Formed in 2015, Q is a growing community, currently with over 2,400 members. Its membership is diverse, including those at the front line of health care, patient leaders, managers, researchers, commissioners, policymakers and others – boosting its power as a source of innovation and problem-solving.

Informed by evidence that flexible networks support innovation and spread, Q enables its members to foster productive connections, including by providing opportunities for them to connect face-to-face and online, for example through local and national events and special interest groups (in areas such as primary care or emergency care).

One Q initiative, the Q Lab, develops this approach further by bringing professionals and patients together to make progress on particular challenges over a fixed period of time. Combining social innovation approaches with health care improvement expertise, the Q Lab provides a space for people to work collaboratively on a problem, develop possible solutions, test new ideas and share learning.

Recent evaluations of Q and the Q Lab have found that members are making useful connections they say they would have been unlikely to make otherwise. Members regularly share stories of borrowing ideas and inspiration from others, enabling them to ‘shortcut’ or improve work they are doing locally. This avoids the wasted effort that comes from people trying to solve problems already addressed satisfactorily by others elsewhere.

Box 9: THIS Institute

The Healthcare Improvement Studies Institute (THIS Institute) aims to create a world-leading scientific asset for the NHS by strengthening the evidence base for improving the quality and safety of health care. Co-created by the University of Cambridge and the Health Foundation, the Institute launched in 2018. It is founded on the guiding principle that efforts to improve care should be based on the highest quality evidence.

THIS Institute is boosting research activity to provide more clarity on what works in improving health care, what doesn’t, and why. By evaluating which interventions work in which contexts, and how they work, the Institute aims to create an evidence base that supports replicable and scalable improvements to health care delivery and patient experiences. Through its innovative fellowship programme, the Institute is also boosting research capacity by creating a new generation of highly trained, multidisciplinary experts with skills in researching health care improvement.

THIS Institute’s work is defined by an inclusive approach that combines academic rigour with the real concerns of patients and staff. It engages a broad coalition from the UK’s wealth of expertise across health care, science and beyond, and works closely with multiple partners from different sectors across the UK, as well as patients and health care staff themselves.

Rethinking the pipeline model

Recognising the role of early adopters as ‘knowledge generators’ challenges the traditional ‘pipeline’ model of innovation, which sees knowledge as generated by the innovator and merely transmitted to others through the diffusion process. As Miller and Shinn put it in the context of health promotion programmes, under a more sophisticated view of innovation and adoption, ‘dissemination becomes not simply the routine application of knowledge developed elsewhere and codified… but the theoretically motivated search for underlying principles of programmes or practices that can inform both understanding of change and programmes to create it’.

This perspective expands on the traditional set of ‘adopter categories’ popularised by Everett Rogers in the 1960s: innovators, early adopters, early majority, late majority and laggards. These were conceived as population types, based on their likelihood of exhibiting certain behaviours, but these categories can also be applied to the diffusion and adoption of a single innovation. In this context, ‘innovators’ are, according to Rogers, the initial adopters of the innovation, early adopters are the next set of people to adopt the idea, and so on, with laggards being the last (see Figure 11).

Models of diffusion rooted in a ‘commercialisation’ paradigm have tended to draw a sharper distinction between pre-market ‘innovation’ stages and subsequent commercial ‘adoption’ stages than is the reality in much process innovation and quality improvement in health care – and, indeed, in other sectors too like software, where developers can iteratively update their product. (Note that the description of adopters as knowledge generators here is different from the notion of ‘lead users’ as innovators, in which the idea and pressure for innovation comes from a group of users, but which is nevertheless still conceived as a ‘pre-diffusion’ stage of the innovation cycle.)

As Figure 11 illustrates, a realistic model of diffusion in health care requires a far more blurred boundary between the idea of the innovator and the initial adopter/early adopter categories. Here we agree with Hawe, who argues that conventional terminology is unhelpful because ‘it privileges a pipeline metaphor of knowledge generation at the expense of understanding and finding ways to convey primary knowledge emanating from practice contexts’ and that ‘a new language is needed that gives expression and legitimacy to models of co-production of knowledge’.

Figure 11: The initial spread process as co-innovation

Case study 2

RIPPLE: How implementation in new contexts can build deeper understanding of the intervention

Chronic obstructive pulmonary disease (COPD) is the second most common cause of emergency admissions in the UK and causes one in 20 deaths. Its symptoms, such as breathlessness, can lead to and amplify anxiety, low self-esteem and social isolation, which in turn can affect mental health and result in poor self-management and lack of engagement with key treatments. In a British Lung Foundation survey, 90% of people with COPD said they were unable to participate in socially important activities.

The RIPPLE clinic (Respiratory Innovation: Prompting Positive Life Experience) was developed in Coventry with the support of a Health Foundation innovation grant in 2014. It is based on the hypothesis that outcomes for people with COPD can be improved by addressing factors such as social isolation, depression and anxiety, rather than simply improving their lung function. In this sense, it is primarily a social, rather than a clinical, intervention. The idea was to take consultations out of a clinical setting and combine them with social activities, such as yoga and bingo, in a way that would reduce isolation for people with severe COPD. The intervention uses a community space to host sessions, given that clinical settings are not always the best environment for engaging patients and facilitating discussions. The session also provides opportunities for people to take part in education and rehabilitation activities. Attendees have reported an increased ability to self-manage compared to before they attended.,

‘For me, the more I’ve got into it, the more I think it’s a huge philosophical change in the way you deal with people with chronic lung disease, actually… I would say for about 95% of the people who attend the group regularly, it’s actually, “This is the only time I get out of the house this week”.’ Interview with innovator

In 2015, the RIPPLE team received a second grant from the Health Foundation, as part of its Spreading Improvement programme, to spread the model to a further six communities in the East and West Midlands. The new sites were given funding, support and advice, as well as opportunities to share their knowledge and experiences.

RIPPLE is a complex intervention, consisting of multiple components. The initial demonstration project had enabled the programme leaders (including the innovator) to begin to identify its core components. These subsequently became framed as a set of ‘pillars’ of the clinic model – for example, activities to reduce social inclusion, transport provision, third sector involvement, exercise activities, mental health support, and so on – and were set out in the ‘Expression of Interest’ document seeking proposals from teams to be part of the Spreading Improvement project.

‘We were really clear that the delivery venue had to be a normal part of a community, and not an NHS building. They had to think about transport as well, because getting people… there easily is one of the big things with social inclusion… We also made sure… that the third sector were… part of the process. So more often than not, it’s the third sector that actually deliver and run these projects, because they know local communities.’ Interview with programme leader

Many aspects of context potentially add to RIPPLE’s complexity. For example, RIPPLE is co-produced with patients, so each clinic’s social activities naturally evolve around patient preferences. Implementation is also affected by the distinct transport requirements and recruitment challenges faced by urban and rural communities. The quality of relationships with other services may also be important, as the intervention brings together partners from organisations with contrasting working cultures and practices. According to the final evaluation report, the model is ‘complex, with no one-size-fits-all [approach] to improving COPD care across varied sites and locations’.

In line with this, the description of the intervention produced by the innovator and programme leader for the ‘Expression of Interest’ document embodied a loosening approach, focusing on the underpinning principles (the ‘pillars’) and goals of the intervention, without prescribing how clinics should be set up. This has given adopter teams a sense of ownership of the intervention, enabling them to adapt it to suit their local context and population group.

‘We wanted to very much co-produce with people with COPD and have that central in the design and implementation, and they [the programme leaders] allowed us to do that and really kind of gave [us] the power.’ Interview with implementation site lead

‘It [the Expression of Interest document] had something briefly about the outcomes but it doesn’t give you any detail about the clinic… I have to say we were kind of given free rein to go from there. As far as policies and anything written, I would say minimal.’ Interview with implementation site lead

‘Self-management… and tackling social isolation were also the key principles, but [it] was loose in terms of how you deliver that.’ Interview with implementation site lead

‘I think you have to go with what people are willing to do locally. It’s very difficult to impose a model on other people because you have to have their engagement and ownership.’ Interview with innovator

The adopter sites have made a range of adaptations to the model. For instance, one substituted the input from a hospital consultant with that of a respiratory specialist, physiotherapists and nurses, because it was economically unviable for a consultant to travel to the clinic. Another added hospice involvement to the model – ‘because actually that was important as far as the COPD patients that we look after’ – as well as input from the fire service to perform safety checks for people going home with oxygen.

The variations and adaptations made to the clinic model during this spread phase are generating useful lessons, improving the innovator’s understanding of the degree to which the intervention can be modified. For example, there have been significant variations in terms of who sets up the clinic: the Coventry clinic was set up by a secondary care physician, but the clinics at the adopter sites have been set up by a range of organisations, including clinical commissioning groups (with clinician input), general practices and third sector organisations. The primary care-led clinic in particular challenged the original model, since patients could self-refer in a way that hadn’t been tested before. The innovator acknowledged that this model caused us both some concern to start with’, but it has since proved highly successful, with preliminary data suggesting that unplanned hospital admissions have been reduced for those attending.

‘I think we have learned from other sites. I like to think of the project as an ongoing, organic thing that grows and changes.’ Interview with innovator

‘… They seem to be very interested in people doing it slightly differently, because again, that always kind of informs potentially how this could be spread further.’ Interview with implementation site lead

The spread phase has also enabled the programme leaders to clarify when variations have departed too far from the original theory of change – for example, when one team wanted to replace an in-person clinic with a virtual clinic.

‘… They thought they could have a sort of Facebook community rather than a real meeting and we had to be quite firm with them about that and [say] there isn’t really any medical input into their model.’ Interview with innovator

More generally, adopter teams have provided valuable learning for the programme leaders by identifying new solutions to particular implementation challenges, as when one site managed to overcome problems accessing a collaborative IT platform.

‘We now know the way round it, we now know the things that if people say “We can’t do that,” we can actually show them, “Well, yes you can”, because it’s been done.’ Interview with programme leader

During the course of the programme, the innovator and programme leader have also begun to learn more about the respective importance of each of the pillars. For example, while the social inclusion pillar remains central to the intervention, they have become less convinced that the physical exercise pillar is as important for the effectiveness of the clinic, based on the experiences of the adopter sites. These experiences have also reaffirmed that hosting the clinics in community settings rather than clinical settings is core to the intervention, but within this constraint they have learned that the clinics can work well in a variety of different settings, from church halls to football grounds.

This Spreading Improvement project concluded in 2018, and the independent evaluation has reported reduced social isolation and some observed improvements in health status for participants. Building on the experiences of the adopter sites, the innovator believes that the next phase of spread could be more specific about how the clinics could be set up, perhaps in the form of a menu of options based on what others have done.

‘I think we could probably produce a handbook now on how to do it. With a sort of menu, so you have some suggestions. You could probably give two or three suggestions under each section, couldn’t you? There would sort of be the main recipe, and then some variations.’ Interview with innovator

Previous Next