About this evaluation and analysis

This evaluation was conducted by the Improvement Analytics Unit – a partnership between the Health Foundation and NHS England that aims to provide local teams with robust information on the impact of initiatives to improve care, in order to inform ongoing improvement efforts. The evaluation was conducted after the end of the first phase of the vanguard implementation, and aimed to provide insights that, when combined with other local evidence on progress made on other objectives, would inform the development and continuous improvement of the services provided by the vanguard. Preliminary findings from the analysis were first shared with Wakefield CCG in April 2018. Analysis was conducted according to a Statistical Analysis Protocol (SAP), which was subject to independent academic peer reviews and developed with and agreed by Wakefield CCG before analysis began.

The evaluation assessed whether the enhanced care package affected hospital use among residents aged 65 or over living in the 15 vanguard nursing and residential care homes in Wakefield during phase 1 of the intervention (14 February 2016 to 19 March 2017). Although the phase 1 care homes continued to receive the enhanced support after March 2017, this study only evaluated the first period, as it was expected that the intervention was likely to have changed as more care homes started to receive the enhanced support.

We assessed the impact of the enhanced care package over and above the effect of other services available in the area, including the enhanced primary care. We compared the hospital use of vanguard residents to a ‘local matched control group’ made up of residents with similar characteristics living in care homes of similar type in Wakefield CCG that were not receiving the enhanced support. Both residents that moved into the care home before the enhanced care package was introduced (‘existing residents’) and those who moved in after (‘new residents’) were included in the study.

This study does not assess the impact of the package on older people’s quality of life, the coordination of care or improving residents’ health or social isolation. These are all areas that the enhanced support was aiming to improve and the value of the enhanced support might relate to improving these other areas in addition to hospital use.

We carried out subgroup analyses to see if there were differences in outcomes between existing and new residents. Many people move into a care home after a crisis such as a fall or an acute illness., New residents would have moved to a new environment and would most likely still have been affected by what made them move into a care home, whereas existing residents would have had time to settle into the care home by the time the study started. By analysing the groups separately, we aimed to get a better understanding of hospital activity at different stages of a person’s care home stay.

Once we had access to the data, we found that about a third of residents were in the study for less than three months. As a sensitivity analysis, we therefore looked at the effect of the care package on vanguard residents that were in the study for at least three months.

Data used in the analysis

The Improvement Analytics Unit has access to pseudonymised data from the Secondary Uses Service (SUS) – a national, person-level database that is closely related to the widely used Hospital Episode Statistics (HES). SUS data contain information on A&E attendances, inpatient admissions and outpatient appointments that are funded by the NHS in England, but unfortunately do not record accurately whether an individual resides in a care home. Therefore, the unit needed a data set containing information on the residents of care homes, and a method of linking that data to the SUS database.

This was assembled by the Arden & Greater East Midlands Data Services for Commissioners Regional Office (Arden & GEM DSCRO), the National Commissioning Data Repository in NHS England and the Improvement Analytics Unit. Care home residents were identified using a combination of monthly care home registration data published by the Care Quality Commission (CQC) and monthly extracts from the National Health Applications and Infrastructure Services (NHAIS) database, which contains a list of individuals registered at each general practice in England, including their residential address and NHS number. For more details see the statistical analysis protocol (SAP).

The linked care home and hospital data were analysed by the Improvement Analytics Unit within an accredited secure data environment based at the Health Foundation. At no point did we have access to identifiable data. Throughout, the minimum amount of data was used for the purposes of the study.

Identifying residents receiving enhanced support

The study focused on individuals in care homes (nursing or residential) that were participating in the enhanced support who met the following criteria:

• had moved into a Wakefield care home between 17 August 2014 and 18 February 2017

• were not known to have previously resided in another care home between 17 August 2014 and their move-in date

• lived in the care home for at least one month during the study period

• were aged 65 years or over

• experienced a hospital admission within the three years before the start of the study (to ensure information on health conditions was available to inform the selection of an appropriate control group).

The evaluation only included nursing and residential care homes located in Wakefield CCG that were open for the entire period between February 2016 and March 2017. Care homes registered as caring for other age groups and with specialties other than the frail, elderly care home population were excluded.

This study comprised 526 residents from the 15 vanguard care homes. This is lower than expected based on the 999 care home beds.

Selecting the matched control group

To evaluate the impact of the vanguard's enhanced support, it was necessary to form a control group of care home residents who were as similar as possible to the vanguard residents.

We formed the control group in three steps: (1) identifying a subset of care homes in Wakefield CCG that were similar to the vanguard care homes but were not receiving the enhanced support; (2) applying the same selection criteria to residents of these care homes as for vanguard residents; (3) selecting residents from the group identified in step two with similar characteristics and living in similar care homes to the vanguard residents.

In step two we identified 37 care homes (eight nursing homes and 29 residential homes) that were similar to vanguard care homes but were not receiving the enhanced support. The same inclusion and exclusion criteria were applied to residents in the control care homes as to those in the vanguard care homes. In total, 625 residents from 30 care homes met the selection criteria and were therefore in the potential control group.

A matched control group that was similar to the vanguard residents on baseline characteristics both at care home and resident level was identified by matching on a range of variables. Variables included individual characteristics such as age, gender, prior health conditions and prior hospital use; and care home characteristics such as type of care home (whether nursing or residential) and number of beds. The variables included in the matching are listed in the SAP.

When selecting control residents, we did not use data on events that occurred after the start of the study, since this could have biased our findings. However, we checked that our final matched control group had a similar mortality rate to the vanguard residents as a check for unmeasured differences between groups. As we did not expect the enhanced care to have a large effect on death rates, a difference in death rates might suggest unmeasured differences between the groups.

We matched vanguard residents in nursing homes with matched control residents living in nursing homes; likewise, vanguard residents in residential homes were matched with residents in control residential homes. One matched control group resident was selected for each vanguard resident, yielding a sample of 526 ‘residents’ living in 30 care homes in the control group. The same resident could be used as a control at each of the two timepoints when care homes could join the intervention (February 2016 and September 2016). Two unique resident records were created for the resident at those time points, reflecting their hospital use and long-term conditions at that time. At each time point, each resident record could be reused three times. Reusing control residents meant that control residents that look similar to many vanguard residents could be selected multiple times, thereby ensuring that the matched control group is more similar to the vanguard group than if control residents could only be selected once. This reduces bias but causes correlation so the width of the confidence intervals is underestimated. We therefore limited the number of times the unique resident records could be reused to three. On average, control resident records were reused twice. Of the 526 ‘residents’ in the matched control group, there were 318 unique resident records. Full details on the methods used are available in the SAP.

Risk adjustment

Comparisons between the vanguard residents and the matched control group were made using multivariable regression analysis. The regression models adjusted, where possible, for differences that remained after matching between the two groups in observed baseline characteristics such as prior hospital use, age and type of co-morbidities. Matching and regression generally perform better in combination than separately. The regression models produced a ‘best estimate’ of the relative difference in the examined hospital utilisation outcome between the vanguard residents and the matched control group, together with a 95% confidence interval. The confidence intervals show some of the uncertainty in the results by providing a range around the ‘best estimate’ in which we can be relatively certain the true value lies. However, the regression cannot adjust for variables that were not recorded in our data sets, such as the degree of family support, social isolation and ability to manage their health conditions. This additional uncertainty is therefore not captured by the confidence intervals, so the results need to be interpreted with caution.

The same procedure was used for the subgroup analyses looking at new residents and existing residents and when looking at residents that were in the study for at least three months.

Figure 2. The process of forming the control group of care home residents and risk adjustment

Source: analysis by the Improvement Analytics Unit

Outcome measures

Once a matched group of control care home residents was satisfactorily formed, the Improvement Analytics Unit proceeded with comparing hospital activity between the two groups. For this purpose, the following eight outcomes were considered for analysis:

• emergency admissions

• emergency hospital bed days

• A&E attendances

• elective admissions

• the subset of ‘potentially avoidable’ emergency admissions, based on a list of conditions considered to be manageable in community settings or preventable through good quality care (see Box 1)

• outpatient attendances

• elective hospital bed days

• deaths occurring in hospital (%) (proxy for not dying in preferred place of death).

Hospital activity was measured for the period from 14 February 2016 until 18 March 2017 during which individuals were residents in a care home. For new residents, this was from the month they moved into the care home and for existing residents from the introduction of the intervention. All residents were followed until the end of the study, they died or moved out of the care home, whichever was earliest. Each resident therefore had a follow-up period of between one and 13 months.

Box 1. Conditions for which we considered emergency admissions to be potentially avoidable

The analysis included conditions that are often manageable, treatable or preventable in community settings without the need to go to hospital, as well as those that may be caused by poor care or neglect. These were:

• acute lower respiratory tract infections, such as acute bronchitis

• chronic lower respiratory tract infections, such as emphysema

• diabetes

• food and drink issues, such as abnormal weight loss and poor intake of food and water, possibly due to neglect

• fractures and sprains

• intestinal infections

• pneumonia

• pneumonitis (inflammation of lung tissue) caused by inhaled food or liquid

• pressure sores

• UTIs.

Potentially avoidable emergency admissions were defined as those hospital admissions where one of these conditions was listed as the primary diagnosis for the admission. Note that the list of conditions was developed by the Care Quality Commission (CQC) as part of their analysis on older people experiencing health and social care and was not therefore designed specifically for care home residents. Note also that sometimes individuals will still need to be admitted to hospital for these conditions independently of the availability of suitable out-of-hospital care (as is perhaps the case with individuals suffering from multiple co-morbidities) and regardless of the quality of the care offered in the care home. The metric is therefore not perfect, but we would expect the enhanced support to show greater impact on reducing the risk of hospital admission for these conditions than for others.

This sensitivity analysis was not specified in the statistical analysis protocol.

Pseudonymised data sets have been stripped of identifiable fields, such as name, full date of birth and address. However, a unique person identifier (such as an NHS number) has been replaced with a random identifier. The scrambled version of that field is used as the ‘key’ to link data sets together. For this analysis, a scrambled version of the NHS number was used to link together hospital records for the same individual over time.

§ The lower bound is due to data availability and the upper bound to allow for a minimum follow-up period of one month.

Elective admissions are defined as those that are ‘ordinary’ or day cases and exclude maternity and regular day/night cases.

** Primary diagnosis for a hospital admission was taken from the first consultant episode of the hospital spell.

Previous Next