About this evaluation

 

This evaluation was conducted by the Improvement Analytics Unit – a partnership between the Health Foundation and NHS England that aims to provide local teams with robust and timely information on the impact of initiatives, in order to inform ongoing improvement efforts (see Box 2).

The report considers data spanning the period from 1 August 2015 (shortly after the new specialist emergency care hospital opened) to 31 July 2016 (1 year later). It therefore considers early data from the PACS programme at a time when the number of departments provided by Northumbria Healthcare offering A&E services to the population of Northumberland CCG had effectively been increased from three to four, but with changes to the clinical model that aimed to centralise major A&E cases, reduce length of stay and improve outcomes.

Due to constraints with national data sets, this report is restricted to considering impacts on hospital use, including A&E visits, inpatient admissions and length of stay. It cannot examine whether there have been improvements in the clinical quality of care (for example, infection rates) or patient outcomes (such as quality of life).

Box 2: The work of the Improvement Analytics Unit

The Improvement Analytics Unit produces robust analyses regarding the impact of changes to how health care is delivered. It achieves this by using large data sets to track the quality of care offered to patients receiving new interventions, and making comparisons against control groups. These control groups are used to provide an estimate of the outcomes that would have been expected in the absence of the initiative being evaluated.

The approach can be understood by drawing a parallel with randomised controlled trials, which are often used to evaluate the impact of new drugs and medical procedures. In these trials, subjects are enrolled and then randomly assigned to receive either the new intervention or more standard care. Outcomes are then assessed at a later point in time – the effect of the intervention is calculated based on the difference in the outcomes of the intervention and control groups.

Randomised controlled trials are often considered the ‘gold standard’ in impact evaluation, since randomisation ensures that two groups are comparable at baseline. Unfortunately, it is rarely feasible to conduct these trials for the kinds of interventions considered by the Improvement Analytics Unit. For example, it would not be possible to recruit large numbers of health care areas and randomly determine which of these should build a new specialist emergency care hospital while converting old A&E departments to urgent care centres.

The unit selects control groups by applying advanced statistical methods to existing NHS data sets. The unit works with academic experts, and for this evaluation worked closely with Stephen O’Neill from the National University of Ireland Galway and the London School of Hygiene and Tropical Medicine, who brought technical expertise in the synthetic control method. The analysis was conducted jointly with NHS England, but this briefing was prepared by the Health Foundation. It is the second from the Improvement Analytics Unit, with the first one considering the impact of enhanced support for people living in care homes.

Previous Next