May 26, 2016

RCTs in the US Legal Profession

D. James Greiner and Andrea Matthews, “Randomized Control Trials in the United States Legal Profession,” Harvard Public Law Working Paper No. 16-06 (2016).

Eighty years ago, one might have been hard pressed to guess which of two quintessential United States professions, law or medicine, would undergo a transformational epistemological change such that the randomized control trial (the “RCT”) would become the gold standard for the evaluation of interventions provided to patients or clients.

We all know what happened. Although the transformation in medicine has (rightly) been criticized as failing to extend far enough, there is no comparing the amount or quality of the objective, scientific evidence available to practitioners in the two professions. And despite a compelling literature on failures within the medical profession to incorporate RCT findings into day-to-day practice, there is no comparing the evidentiary basis for the standard of care as between medical and legal professionals. Practitioners of medicine chose to transform their profession into a science. Practitioners of law did not.

In this essay, we attempt to begin a conversation on the following subject: why is there no domain to review? What can be done to overcome the United States legal profession’s current disdain for the RCT? Relatedly, why did medicine transform itself into a science, while law did not? This essay is our first step at beginning to answer these questions.

In this article, we catalog and briefly analyze those RCTs that do exist in the United States legal profession. Our analysis of these studies focuses not on their scientific content but rather on what they can teach us about the questions outlined above. The intensity of the United States legal profession’s resistance to the RCT is such that, viewed individually, each law RCT appears to be a unicorn, a magical creation with no origin story that appears briefly in a larger setting and then fades away. We expect that analysis of all RCTs in the United States legal profession will reveal generalizable lessons about this pattern of spontaneous emergence and later obscurity.