Annex 2: Evidence about when DSS are most likely to work

The best current evidence for this is Kwan’s 2020 systematic review and meta regression of 122 RCTs studying the impact of DSS in the care of 1.2M patients & 11,000 care providers. It should be noted that this study focuses solely on DSS embedded in electronic health record systems, not the wider range of decision support tools covered by this guide.

This study shows that for DSS embedded in electronic health records, the amount of benefit (expressed as adherence with the intended outcome) varies from 4.4% for prescribing to 6% for vaccination and around 7% for DSS that support test ordering, documentation and “other”, which included diagnosis, guideline adherence and specialty referral - see Table 10 below.

 

Table 10: Results of Kwan’s analysis of DSS impact by type of outcome [redrawn from Kwan 2020]

Type of outcome

Number of trials

Total number of patients

% improved adherence (95% CI)

Prescribing

68

731726

4.4% (2.6 to 6.2)

Test ordering

30

372640

6.8% (4.9 to 8.6)

Documentation

25

403161

7.1% (5.4 to 8.9)

Vaccination

12

75731

5.9% (4.1 to 7.7)

Other (diagnosis, guideline adherence and specialty referral)

35

391959

6.8% (5 to 8.6)

All trials

122

1203053

5.8% (4.0 to 7.6)

A minority of studies achieved substantial increases in the delivery of recommended care, but predictors of these more meaningful improvements remain undefined.

An earlier analysis of 70 different studies (20 randomized controlled trials, 33 prospective nonrandomized studies with pre/post analysis or cohort studies and 18 retrospective studies) generated a list of 2 clinical tasks and 6 conditions in which DSS seem to most effective, with large effect sizes and low to medium risk of bias [Varghese 2018]:

  • Managing blood glucose and blood transfusions
  • Preventing physiological deterioration, pressure ulcers, acute kidney insufficiency (AKI) and venous thrombo-embolism (VTE)

Overall, across all 70 studies, 70% showed clinical benefit and 29% showed no change (Varghese 2018).