A workplace app halved distress scores, and lost a fifth of the people using it

Design

Randomised controlled trial, four weeks, simple randomisation, unblinded

Population

201 working adults randomised (102 intervention, 99 control); 178 analysed

Primary outcome

DASS-21 total score after four weeks

Effect

Mean change -14.518 intervention vs +3.319 control (F 1,176 = 59.358, P < .001, eta-squared 0.252), driven mainly by the stress subscale

Two hundred and one working adults were randomised to a four-week smartphone intervention combining cognitive behavioural, mindfulness and self-regulation exercises with ecological momentary assessment - prompts delivered in the moment rather than in a weekly session - or to control. The outcome was the 21-item Depression, Anxiety and Stress Scale.

The difference is large. DASS-21 fell by a mean of 14.5 points in the intervention group and rose by 3.3 in controls (F 1,176 = 59.358, P < .001, eta-squared 0.252). Stress carried most of it. Engagement was modest: a mean of 6.27 exercises completed, 9.74 mood journal entries, and just under five minutes a day. Notably, the amount of use was not associated with the amount of improvement.

The problem sits in the dropout. Twenty-one of 102 intervention participants left the study against 2 of 99 controls, a difference the authors themselves flag as highly significant. The 178 people analysed are therefore the ones for whom a daily prompting app was tolerable, compared against a control group that had nothing to tolerate. That is not the same comparison as the one the headline result implies. For a psychiatrist asked whether an employer's wellbeing app is worth endorsing, the honest answer is that it appears to help the people who stay with it, and a fifth did not.

• Ask employers presenting app data what their dropout rate was, and in which arm
• Do not offer a workplace app as a substitute for assessment in someone with a probable disorder
• DASS-21 measures distress in non-clinical populations; it is not a diagnostic instrument
• Five minutes a day was the observed engagement - set expectations at that level, not at daily practice
• Dose did not predict response here, which argues against telling patients to use it more

Why it matters

Employers are buying these; the question in clinic is what to say when a patient's workplace has issued one.

Don't overread it

Differential dropout means the analysed sample is self-selected for tolerating the intervention.

The statistics, in plain English

An eta-squared of 0.252 means about a quarter of the variation in outcome is attributable to group allocation, which is a very large effect for a four-week self-help intervention and should prompt scrutiny rather than enthusiasm. Two features explain much of it. The control group got worse, so the gap is the sum of improvement in one arm and deterioration in the other. And differential dropout of 21 against 2 means the analysed groups are no longer the randomised groups - the comparison has quietly become people who tolerated the app versus everyone assigned to control. The absence of any dose-response relationship is a further reason for caution: if the exercises were driving the benefit, doing more of them should have helped more.

Read the rest in the app

You have read your two free briefings this month. The app carries all 27 specialties, every morning, free — and this finding is waiting in it.

Get Daily Doctor — free

Scan to keep reading on your phone. No account needed to start.