Who Evaluates Digital Health? The Developer Effect

A new npj Digital Medicine rapid review examined the “developer effect” in trials of Digital Health interventions. Across 229 randomized trials drawn from 29 systematic reviews, 73% involved the intervention’s developer; only 27% were conducted independently.

Developer-involved trials were more likely to report statistically significant results. When the analysis was weighted by sample size, they had 23% higher odds of reporting such results than independent trials. They were also more likely to be preregistered.

What the result means

This association does not show that developers manipulated studies or inflated treatment effects. Developer-involved trials may have different resources, designs, registration practices or intervention maturity. The review calls for closer examination of how different forms of developer involvement affect trial outcomes.

My takeaway

A well-designed product and a randomized trial can create a persuasive impression of evidence. We should also know who designed, conducted, analysed and funded that trial. Developer-led validation is useful first evidence; independent replication provides a stronger basis for decisions about widespread reimbursement and implementation.

Real-world effectiveness studies and monitoring after deployment should continue the evidence chain as digital interventions change. The speed of product updates makes continuing scrutiny especially important.