Evaluation of the NHS England evidence-based interventions programme: a difference-in-difference analysis

Abstract

Background The NHS England evidence-based interventions programme (EBI), launched in April 2019, is a novel nationally led initiative to encourage disinvestment in low value care. Method We sought to evaluate the effectiveness of this policy by using a difference-in-difference approach to compare changes in volume between January 2016 and February 2020 in a treatment group of low value procedures against a control group unaffected by the EBI programme during our period of analysis but subsequently identified as candidates for disinvestment. Results We found only small differences between the treatment and control group after implementation, with reductions in volumes in the treatment group 0.10% (95% CI 0.09% to 0.11%) smaller than in the control group (equivalent to 16 low value procedures per month). During the month of implementation, reductions in volumes in the treatment group were 0.05% (95% CI 0.03% to 0.06%) smaller than in the control group (equivalent to 7 low value procedures). Using triple difference estimators, we found that reductions in volumes were 0.35% (95% CI 0.26% to 0.44%) larger in NHS hospitals than independent sector providers (equivalent to 47 low value procedures per month). We found no significant differences between clinical commissioning groups that did or did not volunteer to be part of a demonstrator community to trial EBI guidance, but found reductions in volume were 0.06% (95% CI 0.04% to 0.08%) larger in clinical commissioning groups that posted a deficit in the financial year 2018/19 before implementation (equivalent to 4 low value procedures per month). Conclusions Our analysis shows that the EBI programme did not accelerate disinvestment for procedures under its remit during our period of analysis. However, we find that financial and organisational factors may have had some influence on the degree of responsiveness to the EBI programme.