A sphincter injury model that calibrates well and discriminates moderately

Design

population-based cohort study with model development, internal and temporal external validation

Population

171046 nulliparous women with spontaneous vaginal delivery and 37547 with operative vaginal delivery, singleton cephalic at term, Netherlands 2016 to 2020

Primary outcome

obstetric anal sphincter injury

Effect

injury rate 4.1% and 3.5%; area under the curve 0.67 (95% CI 0.67 to 0.68) and 0.68 (0.67 to 0.70); Brier scores 0.039 and 0.032

Using the Netherlands Perinatal Registry, this study built and temporally validated prediction models for obstetric anal sphincter injury in nulliparous women delivering a singleton at term in cephalic presentation between 2016 and 2020 - separately for spontaneous and operative vaginal delivery.

Among 171046 spontaneous deliveries the injury rate was 4.1%, and ten predictors were retained: mediolateral episiotomy, expected fetal birth weight, duration of the second stage, occipitoposterior position, induction of labour, epidural analgesia, Asian ethnicity, maternal age, gestational age and fetal sex. Among 37547 operative vaginal deliveries the rate was 3.5% and seven predictors survived. Both models had moderate discrimination - area under the curve 0.67 (95% CI 0.67 to 0.68) and 0.68 (0.67 to 0.70) - and excellent calibration, with Brier scores of 0.039 and 0.032.

That combination is worth understanding rather than glossing. Excellent calibration means the predicted risks are close to the observed ones on average, so a woman told her risk is 8% really does face about 8%. Moderate discrimination means the model is poor at separating the individual who will be injured from the one who will not. So this is a counselling and planning instrument, not a triage tool. It also includes Asian ethnicity as a predictor, which reflects an association observed in Dutch registry data and not a mechanism - treat it as a marker whose meaning is unresolved rather than as a biological finding, and be careful about how it is presented to a woman.

• Use the model for counselling and planning, not to decide who is safe - discrimination is moderate.
• Its predictors are the modifiable ones you already discuss: episiotomy, second stage duration, position, epidural.
• Excellent calibration means quoted risks are trustworthy in aggregate; it does not mean the model identifies individuals.
• Asian ethnicity as a predictor reflects a registry association and not an established mechanism; handle it with care in a consultation.
• The models were built on Dutch practice, where mediolateral episiotomy is standard - transfer to settings with different episiotomy practice is untested.

The statistics, in plain English

Calibration and discrimination measure different things and this pair separates them neatly. A Brier score of 0.039 with an event rate of 4.1% indicates predictions that track observed risk closely across the population. An area under the curve of 0.67 means that given one woman who sustained an injury and one who did not, the model ranks them correctly about two-thirds of the time - barely better than useful, and typical of obstetric outcome models where most of the variance is not captured by recorded variables. Temporal external validation, testing the model on a later period rather than a random split, is the more demanding check and is what makes these figures credible.

Read the rest in the app

You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.

Get Daily Doctor — free

Scan to keep reading on your phone. No account needed to start.