Why Randomized Controlled Trials (RCTs)?

Pt. 1: Robot Babies

Welcome! This is the first post in a series called “Why Randomized Controlled Trials (RCTs)?” which will explore experiences across disciplines. Here we explain how RCTs have often turned commonly accepted beliefs on their heads and reveal how rigorous testing is needed even when, and perhaps especially when, we assume to know the best practices in a field.

A staple of TV shows and popular culture that focus on sex ed is the cliché of students taking care of a sack of flour or an egg to simulate the time-consuming elements of parenthood, in an effort to combat teenage pregnancy rates. Well, the modern version of this has become robot babies, or infant simulators, popular in programs across the world. These babies seem disturbingly real- they cry; they, err . . ., do what babies do frequently; they need attention; and they are made to look as human-like as possible. The program has been received positively by students, health practitioners, and community members alike and many are convinced that it strengthens community bonds.  Incidentally, each baby costs thousands of dollars.

However, results of an RCT in Western Australia have revealed that robot baby programs may in fact be doing the opposite- leading to more teenage pregnancy rather than less. In the Western Australia RCT, the proportion of girls giving birth and proportion of girls getting abortions was higher in the intervention group (the group that received normal sex-ed plus the infant simulator program) than in the control group (the group that received normal sex-ed without the infant simulator program).  Modeling results were statistically significant (although there might have been issues with dropouts).  At a minimum, there is little evidence to suggest that the expensive robot babies decreased teen pregnancies.

So while observational studies led to positive reviews of a program meant to curb teen pregnancy, when tested rigorously, the expensive program was at best doing little and at worst leading to more pregnancies. Without an RCT (and even despite it) this program could continue under the best intentions and assumptions, while in fact encouraging the exact behavior it seeks to curb. We can’t assume we know what works and what doesn’t- we need rigorous testing if we want to make positive change.

What do we in law “know” to be “working” right now?