• The NHS is developing new ways of delivering health care, yet local teams often lack access to the analytical skills they need to judge whether improvements are being made to the quality of the care received by patients. To help meet this need, the Health Foundation has partnered with NHS England to establish the Improvement Analytics Unit. The unit will feed back information on a regular basis to teams participating in transformation programmes in England.
• This briefing represents the first output from the Improvement Analytics Unit. It concerns the impact of providing enhanced support for older people living in 22 care homes in Rushcliffe, and in one care home in a neighbouring area of Nottinghamshire. The enhanced support was developed by Principia, a local partnership of general practices, patients and community services. The package included aligning care homes with general practices, regular visits from a named GP, improved support from community nurses, independent advocacy and support from the third sector, and a programme of work to engage and support care home managers.
• The Improvement Analytics Unit examined the impact of the enhanced support on hospital utilisation using newly developed linked care home and hospital data. Residents of Principia care homes were compared with a matched comparison group, consisting of similar individuals living in similar care homes in comparable areas of England.
• Principia care home residents attended accident and emergency (A&E) departments 29% less often than the matched comparison group, and were admitted to hospital as an emergency 23% less frequently. There was no evidence of an impact on the number of nights a person spent in hospital, elective admissions or outpatient attendances. Residents in the Principia care homes were just as likely to die outside of hospital as matched comparison residents.
• Assuming the two groups were comparable, the most likely explanation of the Improvement Analytics Unit's findings is that they reflect higher quality care for residents of the Principia care homes. The matched comparison group had similar age, gender, health conditions and prior hospital utilisation to the Principia residents, although they may have differed in unobserved ways.
• Before the approaches used are replicated elsewhere, it is important to understand the causes of the lower observed use of emergency care. It is not possible to tell from this analysis what might have been the ‘active ingredient(s)’ leading to the lower rates. This could be related to the enhanced support, some other aspect of care delivery or local context. A process evaluation (looking at the context, implementation and how the package might have impact) would help to identify the active ingredients.

To improve patient outcomes in an increasingly challenging financial climate, the NHS is developing new ways of delivering care. Some of these initiatives require cross-organisational working, blurring traditional distinctions between primary, secondary, physical health, mental health and social care.

To be effective, improvement efforts require access to robust and timely information on the differences that changes are making to the quality of the care provided to patients. Unfortunately, local teams often lack access to the analytical skills and data they need to judge whether improvements are being made. To help meet this need, the Health Foundation is partnering with NHS England to establish the Improvement Analytics Unit. The new unit is working with local teams in England that are participating in national transformation programmes, and feeding back information on progress against key metrics related to the quality and efficiency of health care.

The ambition is to establish the Improvement Analytics Unit as a resource within the NHS. It began with two initial pilots to test the technical feasibility of the approach to linking data and selecting control groups. This briefing is the first output and considers the impact of providing enhanced support to people living in care homes in the NHS Rushcliffe Clinical Commissioning Group (CCG) area. Since it was not possible to randomise individuals to alternative interventions, comparisons were made against retrospectively matched control groups. The analysis was conducted jointly by the Health Foundation and NHS England. This briefing is authored by the Health Foundation and considers the findings of the analysis.

While the analysis provides useful information regarding the effect of enhanced support for care home residents in Rushcliffe, ultimately the greatest value will come from combining the quantitative evidence from the Improvement Analytics Unit with other evidence, such as from qualitative evaluation, to guide the progress of new models of care or other projects as they evolve. Thus, in the longer term, initiatives like the Improvement Analytics Unit might form one part of a wider approach to ‘rapid-cycle evaluation’, in which qualitative and quantitative information is combined with quality improvement skills to enable ‘course correction’ in a more timely manner than has been possible before now.

The study was focused on individuals who:

• moved into a Principia care home (including one from outside the Rushcliffe area) that cared specifically for frail older residents between August 2014 and July 2016
• were not previously residents of care homes (as far as we could detect in our data)
• were aged 65 years or over
• experienced an inpatient admission to hospital within the two years before moving to the care home.

Although the enhanced support was introduced in April 2014, the study only examined people who moved into a care home from August 2014. This was partly because the NHAIS extracts began in August 2014, but we were also aware, from our conversations with Principia, that it took several months for the enhanced support to become fully operational. Thus, the approach allowed for a five-month ‘bedding in’ period for the intervention.

Individuals without a history of inpatient admissions were excluded as we lacked the information on their health conditions that was needed to select a comparison group. We included residents regardless of whether they changed to the aligned general practice, since they may have benefited from the enhanced support regardless. This meant we examined 588 residents from 23 care homes that participated in the enhanced support programme.

Rushcliffe had relatively low rates of hospital admission before the enhanced support was introduced. Across the entire population (not just care home residents), in 2013/14, there were 776 emergency admissions for every 10,000 individuals living in Rushcliffe, compared with 891 for the average local authority in England. Compared with other areas of England, Rushcliffe also had relatively low levels of socioeconomic deprivation and low population density. Therefore, the Improvement Analytics Unit selected comparison residents from areas that had similar demographic and socioeconomic characteristics and emergency admission rates to Rushcliffe: Harborough, Blaby, Test Valley, South Cambridgeshire, Chelmsford and Brentwood. To the best of our knowledge, none of these were offering area-wide interventions like Principia’s enhanced support during the period concerned. The technical appendix contains more information on how the areas compared with Rushcliffe.

We applied the same inclusion and exclusion criteria to care home residents in the six comparison areas as we had to Principia residents. This produced a total of 2,957 residents aged 65 years or over from 93 care homes. From that wider group of care homes and residents, a matched subset of care homes and residents was identified that was similar to the Principia group on the following characteristics.

• Care home characteristics: The number of beds in the care home; whether the care home was registered with the CQC as a nursing or residential home; whether the care home was registered as caring for additional population groups in addition to older people; whether the care home was in a rural or urban setting; the socioeconomic deprivation level of the local area.
• Resident characteristics: Age; gender; ethnicity (white or non-white); particular health conditions associated with frailty (such as cognitive impairment, falls or significant fractures); particular health conditions that have been shown to be predictive of emergency hospital readmissions (such as chronic pulmonary disease and congestive heart failure); an index based on conditions that predict subsequent mortality (the Charlson index); and the number of emergency admissions, potentially avoidable admissions, nights in hospital, A&E attendances, elective admissions and outpatient attendances prior to moving to the care home.

The matched comparison residents were selected using genetic matching, which is a computer-intensive search algorithm that often produces more closely balanced groups than other methods. The approach ensured that the Principia and matched comparison residents moved into care homes at similar points in time, without seasonal differences in those admission dates.

We selected one matched comparison resident for each Principia resident, giving us 588 residents in the comparison group from 64 care homes.

Once the Improvement Analytics Unit was content that the matched comparison group was similar to the Principia residents, the unit proceeded to compare how often residents used hospital care. The following seven impact measures were assessed.

• Attendances at A&E departments (which might not result in a hospital admission).
• Emergency admissions (occurring through A&E departments, or via direct and urgent referrals from GPs and other health care professionals).
• The subset of ‘potentially avoidable’ emergency admissions, based on a list of conditions considered to be manageable in community settings or preventable through good quality care (see Box 2).
• Elective admissions.
• Outpatient attendances.
• The number of nights spent as a hospital inpatient, following either an emergency or elective admission. We expressed this as a percentage of the total length of stay in the care home, and excluded admissions where the person was admitted and discharged on the same day.
• The percentage of deaths that occurred outside of a hospital. We included this metric because the enhanced support included an element of end-of-life planning.

Hospital utilisation was measured for the period during which individuals were resident in care homes, counted from the month during which they moved into the care home to the month they departed. We tracked hospital utilisation until August 2016, so the analysis considered between 1 and 23 months of utilisation, depending on when the person joined (and left) the care home. On average, we followed Principia residents for 211 days (standard deviation 178 days), and matched comparison residents for 201 days (standard deviation 177 days).

Comparisons between the Principia residents and the matched comparison group were made using multivariable regression. The regression models were not strictly necessary, since the matched comparison and Principia residents already had similar baseline characteristics. However, some small differences remained even after matching, and the regression models adjusted for those. Matching and regression generally perform better in combination than separately. The regression models produced a ‘best estimate’ of the relative difference in hospital utilisation between the Principia residents and the matched comparison group, together with a 95% confidence interval.

The analysis was conducted in line with a statistical analysis plan, which was discussed with a Technical Advisory Group and finalised before the data analysis began.

After matching, the matched comparison group was similar to the Principia residents across the range of individual and care home characteristics considered. Full information is available in the technical appendix, but for example Principia residents were aged 85.9 years on average, compared with 86.6 years for the matched comparison group. The two groups also had a similar profile of health conditions (see Figure 2). For example, 53% of both Principia and matched comparison residents were recorded as having dementia, while mobility problems were noted for 25% of the Principia residents and 21% of the matched group.

Figure 3 shows trends in hospital utilisation. The left-hand side of each chart shows that the Principia and matched comparison residents experienced similar rates of hospital activity before moving into care homes. This is to be expected due to the matching process. However, some differences still existed for elective admissions and outpatient attendances. The right-hand side shows that, following moving to the care home, the Principia residents appeared to use less hospital care than the matched comparison group for certain types of hospital activity. The regression modelling aimed to quantify how different these patterns were, when adjusting for the residual differences between the two groups in prior hospital use, as well as other baseline characteristics.

After moving to a care home, Principia residents experienced 0.74 A&E attendances each year on average, compared with 1.02 for the matched comparison residents. After regression adjustment, Principia residents attended A&E departments 29% less often than the matched comparison group. The 95% confidence interval suggested a relative difference in the range 11% to 43% (see Table 1).

Therefore, notwithstanding statistical uncertainty, it seemed that the Principia group experienced fewer A&E attendances than the matched comparison group.

Figure 2: Percentage of Principia and matched comparison residents with certain health conditions

Figure 3: Number of hospital contacts per quarter for the Principia and matched comparison residents

Note: this table compares hospital utilisation in the period following moving to a care home. It is based on an average length of stay of 211 days (standard deviation 178 days) among the Principia residents, and 201 days (standard deviation 177 days) for the matched comparison group. For this Table and Table 2, we excluded residents who remained in a care home for under one month, and thus studied 568 Principia residents and 565 matched comparison residents. The adjusted rate ratios were obtained by applying regression models to the matched data. Note that, following adjustment, the Principia group experienced more elective admissions than the matched comparison residents, even though the unadjusted rate was lower. Since the confidence interval was very wide, our analysis for elective admissions was inconclusive.

The Principia group experienced an average of 0.64 emergency admissions per person for each year in the care home, compared with 0.78 for matched comparison residents. After adjustment, Principia residents experienced 23% fewer emergency admissions than the matched comparison group. The 95% confidence interval spanned 3% to 39% so, again, it seemed that there were fewer emergency admissions for Principia residents than the matched comparison group.

Principia residents experienced 0.22 potentially avoidable admissions per person per year, compared with 0.30 for matched comparison residents. After adjustment, there were 28% fewer potentially avoidable admissions among the Principia group. This is, proportionately, a greater change than detected for all emergency admissions (28% compared to 23%), but greater uncertainty existed in the figures for potentially avoidable admissions. For these, the 95% confidence interval spanned 0% (no difference) to 49% fewer admissions. As a result, it was not possible to be confident that the Principia group experienced fewer potentially avoidable admissions than the matched comparison residents.

There was no evidence for a difference in elective admissions and outpatient attendances. On average, the Principia residents spent 3.2% of nights as a hospital inpatient (see Table 2), compared with 4.1% for the matched comparison group. Although Principia residents had 14% lower odds of being an inpatient on any given day, there was substantial uncertainty, with a very wide confidence interval that included the possibility of there being no difference or Principia residents having higher odds. Therefore, it is not possible to conclude that the two groups differed in terms of nights spent in hospital. Also, during the period examined, around one-third of each group of residents died. Among Principia residents, 81.5% of deaths occurred outside of hospital, compared with 83.5% for the matched comparison group. However, after adjustment, Principia residents had 7% greater odds of dying outside of hospital than the matched comparison group, but the confidence interval was very wide and again the analysis was inconclusive.

Note: The adjusted odds ratios were obtained by applying regression models to the matched data. Following adjustment, the Principia group was more likely to die outside of hospital than the matched comparison group, even though a smaller percentage of them died outside of hospital. Our analysis for both metrics was inconclusive, due to the wide confidence intervals.

Older people living in care homes that participated in the enhanced support programme attended A&E departments 29% less often than a matched comparison group (95% confidence interval: 11–43%), and were admitted to hospital as an emergency 23% less frequently (95% confidence interval: 3–39%). These findings require careful interpretation, informed by a discussion of the strengths and limitations of the analysis.

In the absence of randomisation, residents of the participating care homes were compared with a retrospectively matched comparison group. These two groups had similar age, gender, health care conditions and previous hospital utilisation, and they lived in care homes that were similar in terms of size and provision of nursing care. Moreover, the matched comparison residents lived in local authority areas that were comparable to Rushcliffe in terms of demographic and socioeconomic characteristics and historic, per capita rates of emergency hospital admission. Notwithstanding these similarities, unmeasured differences may have existed between the Principia residents and the matched comparison group, for example in the availability of informal care support or local authority funding. Also, although we ensured that the comparison areas had similar overall emergency admission rates to Rushcliffe, we could not assess whether they had similar levels of admissions from care homes in the period running up to the introduction of the Principia enhanced support in April 2014. These unmeasured differences might explain some or all of the difference we observed in the utilisation of emergency care. However, although we cannot be definitive, some reassurance is available. For example, the Principia and matched comparison groups had similar mortality rates, lending support to the notion that neither group had more severe health conditions than the other. In fact, the two groups appeared similar even before matching, which might mean there was limited scope for bias through unobserved variables. More information is available in the technical appendix.

If the lower use of emergency care does not reflect differences in the characteristics of the individuals, care homes or areas, then the most likely explanation is that it reflects the care provided to care home residents. However, we could not determine definitively whether the enhanced support produced the lower levels of emergency care utilisation, or some other innovation in care delivery. Here, it would help to have data on admissions from care homes before April 2014. This would enable comparisons to be made between new residents in Principia care homes and people who moved to the same care homes earlier, before the start of the enhanced support, in a ‘difference-in-difference’ style analysis. Such analysis would reveal whether the reductions in emergency care utilisation occurred at the same time as the introduction of the enhanced support, potentially leading to stronger evidence about attribution.

It would also be helpful to combine quantitative with qualitative evaluation. Based on our own discussions with Principia, we have described the enhanced support as including regular visits from a named GP, improved support from community nurses, independent advocacy and support from the third sector, and a programme of work to engage and support care home managers. However, we could not observe how these elements operated in practice, or examine the mechanisms through which the enhanced support might have interacted with contextual factors to produce the lower rates of A&E attendance and emergency admission. Such insights would be helpful because complex interventions such as these often evolve over time in response to experience and changing priorities and needs. Understanding these dynamics is important when considering whether to replicate the enhanced support, since the effects might be different in other contexts. Certain elements of the enhanced support package may be more significant determinants of emergency care utilisation than others.

The effect of the enhanced support package might vary according to the characteristics of the residents involved. By using linked data sets, the Improvement Analytics Unit could examine both local authority and privately funded residents. However, the analysis was restricted to residents aged 65 or over who moved to a care home following the introduction of the enhanced support. Therefore, it did not consider impacts on existing residents. Furthermore, the unit considered only people who experienced at least one inpatient hospital admission in the two years before moving to the care home.

This analysis focused on hospital care. This is an important consideration for people with social care needs, who report fragmentation in services, with frequent handovers between care teams often leading to duplication and confusion. However, lower admission rates do not always indicate better patient outcomes and lower cost. Therefore, further analysis is needed regarding impacts on clinical metrics, patient outcomes and the cost of care (which should include the cost of providing the enhanced support to care home residents). Further data linkage, beyond that already performed, might provide a fuller picture. This could draw on information recorded by local authorities when assessing individuals for social care support, or on clinical diagnostic data recorded in the primary care electronic medical record. Ideally, digitally mature health and social care systems would begin to record the outcomes reported by individual service users on a routine basis – this information could then inform improvements to service delivery, as well as direct care. Qualitative studies might also explore the perceptions and experiences of care home residents, their families, and health and social care practitioners.

The Improvement Analytics Unit is a partnership between the Health Foundation and the Operational Research and Evaluation Team at NHS England.

This briefing was authored by Therese Lloyd, Arne Wolters and Adam Steventon from the Health Foundation, based on analysis undertaken by the Improvement Analytics Unit. The wider Improvement Analytics Unit team also includes Sarah Blundell, Richard Brine, Martin Caunt, Stefano Conti, Caroline Gori, Carlotta Greci, Creina Lilburne, Stephen O’Neill, Filipe Santos, Florence Starr and Charles Tallack.

The Improvement Analytics Unit reports to an Oversight Group and has established a Technical Advisory Group. We are grateful to members of these groups for their support and advice, as well as to Fiona Callaghan, Elizabeth Harris and Robert Taylor from NHS Rushcliffe Clinical Commissioning Group for their continued support.

Errors or omissions remain the responsibility of the authors alone.