Why Randomized Controlled Trials (RCTs)?

Pt. 3: Scared Straight

This is Part 3 in a series called “Why RCTs?” which explores experiences with and without the benefit of randomized study across disciplines. You’ve read about robot babies and hormone replacement therapy.  We now bring you an example from the criminal justice system that will show you the possible consequences of implementing programs without knowledge of their effectiveness- even when we have a good gut feeling of how those programs should work.

Maybe some of you have heard of “Scared Straight” programs.  Popular in the 80s (and still today), these are criminal justice programs intended to divert at-risk youth away from the type of behavior that would land them in jail. Essentially these kids are taken on a tour of a detention facility where they see how horrible it is inside the jail.  In some versions, the inmates yell and harass the students. The program is meant to “scare straight” these kids and let them know what is at stake if they don’t shape up and make bad decisions. Two PBS documentaries were aired in 1979 and 1999 that reported high success rates for “Scared Straight” programs, with almost all of the participants staying out of trouble for three months after attending the program.

But really, it’s these programs that should scare is us. Several RCTs suggest that “Scared Straight” do not divert kids from criminal activity.  Rather the programs have no effect, or more likely, increase recidivism rates.  Despite the documentaries that showed high success rates, a systematic review of 9 RCTs on the program found that “scared-straight” programs actually increased crime between 1-28% compared to the no-treatment control group. While each of these RCTs had a no-treatment control condition, with at least one outcome measure of “post-visit” criminal behavior, the observational studies in the documentaries did not provide any data on control or comparison groups. In addition, those chosen to participate in the program were not randomized.  Selection bias, anyone?

Perhaps we should be “scared-straight” when non-rigorous evaluations are used to evaluate the success of a program, leading to its implementation and a general culture of belief in its success, a success that has yet to be proven.