First, a confession: flossing and I enjoy a complicated relationship. I do it, but not as consistently as four out of five dentists might prefer. Yet, using several other dental hygiene implements, I somehow emerge from biannual checkups with pretty solid marks. And it seems I have (the absence of consistent) science on my side!
One prominent meta-analysis of twelve RCTs from 2011 found only weak evidence that brushing and flossing reduce plaque, that awful substance that breeds gingivitis and other oral maladies. Moreover, the low quality of the experimental designs used precluded any definitive inference on tooth decay outcomes. That flossing can reduce gum bleeding was the only realistic conclusion the researchers could draw. Additional evidence gathered over the last decade has steadily hammered final nails in the unqualified pro-flossing coffin. Even the U.S. Department of Health and Human Services no longer promotes the practice.
If all of the evidence on flossing is at best conflicting, what to make of Jamie Holmes’ op-ed in Sunday’s New York Times? His primary target is the RCT itself. Writing “[f]or one thing, it’s unlikely that an Institutional Review Board would approve as ethical a trial in which, for example, people don’t floss for three years. It’s considered unethical to run randomized controlled trials without genuine uncertainty among experts regarding what works,” Holmes cites the correct criterion but then erects stark blinders around the “genuine uncertainty” in the flossing literature. His faith in individual dentists’—and presumably other experts’—received, observational wisdom is sufficient to disregard or even discard several meta-studies. The contradiction continues when he states that “randomized trials are ideal for evaluating the average effects of treatments,” but that “such precision isn’t necessary when the benefits are obvious or clear from other data.” Where are these clearly obvious data? They simply don’t exist as the analyses noted above prove. The strongest such evidence might only be a sense impression that particular patients, who might (like me) either misrepresent their flossing behavior or handle plaque buildup in other ways, benefit from threading between their teeth.
Thus equating the upside of flossing to the advisability of using parachutes at high altitudes doesn’t carry much weight. The only practices that don’t need to be subjected to RCT evaluation are the rare genuine slam dunks, the simple choices that no one ever would seek to justify with rigorous evidence. But even if we were sure of an almost tautological intervention (e.g., wearing a heavy coat to stay warm during Cambridge winters), there is still another argument to consider. What if we knew, to take another example Holmes mentions, that providing glasses to children with eyesight trouble leads to better educational outcomes? An RCT could still be useful because there might be a less expensive alternative available. What if students could be arranged in the classroom according to their vision needs or materials were devised that were easier for them to read? One could design an ethical RCT to provide preliminary answers. We’ve learned several lessons from Sophie’s “Why RCTs?” series, primarily that relying on observation alone can lead policymakers disastrously astray. We also know that, as the flossing meta-studies reinforce, not all RCTs are created equal. (See, in particular, her inaugural post.) But that is no reason to disparage the methodology so strongly.
In short, the problem with Holmes’ argument is that he doesn’t offer a better method of distinguishing between two sets of interventions: (1) practices that are so obviously effective and (2) sacred cows that experts in a field “know” work. The latter might in fact not be effective at all, or might not be any better than less expensive or less invasive or less time-consuming alternatives. The RCT is precisely that superior method, but Holmes seems to belittle the movement promoting its use as a “cult.” We at the A2J Lab, which is at the forefront of producing RCT-based knowledge in the law, take a much different view. Expert opinion matters, but we are less confident that professionals can always distinguish their widely held priors from indisputable truth. We hardly think that RCTs should be “fetishized.” Rather, we see it as a tool toward developing—not impeding—more testable and usable knowledge.
And you can bet that I will continue to be a subpar flossing patient.