Guideline-supporting perioperative trials are often fragile: a few events could tip the result

Design

Methodological survey of randomised trials cited in perioperative guidelines (2012 to 2022)

Population

161 superiority trials (from 639 eligible of 1,868 identified), median sample size 120

Primary outcome

Fragility index, reverse fragility index and fragility quotient

Effect

Median fragility index 4 (IQR 2 to 8); cardiovascular 6, paediatric 1

This methodological survey looked at randomised trials cited in North American and European perioperative guidelines published between 2012 and 2022. Of 1,868 trials identified, 639 were eligible and 161 superiority trials were analysed.

The fragility index is the number of patients whose outcome would have to change to make a significant result non-significant. The median across these trials was 4 (interquartile range 2 to 8) with a median sample size of 120. Cardiovascular trials had the highest median (6) and paediatric trials the lowest (1). Obstetric trials had higher values than adult trials (incidence rate ratio 1.63), and single-centre trials lower than multicentre (0.52).

The authors say fragility should complement, not replace, other measures of trial quality and certainty of evidence. For a clinician it is a reason to be cautious about a recommendation that rests on a single small trial.

• Look at event numbers, not only P values, when a trial supports a practice change.
• Be more cautious with small single-centre trials, which were the most fragile.
• Give more weight to results replicated across centres.
• Do not discard a guideline on fragility alone; it is one measure among several.

Why it matters

Many recommendations in anaesthesia rest on trials whose conclusions depend on a few patients.

Don't overread it

The fragility index is one metric and was calculated for the outcome supporting each recommendation; it does not show that any specific guideline is wrong.

The statistics, in plain English

A fragility index of 4 means that changing the outcome of four patients, for example four more events in one arm, would remove statistical significance. It does not say the treatment does not work, only that the result is numerically unstable.

Read the rest in the app

You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.

Get Daily Doctor — free

Scan to keep reading on your phone. No account needed to start.