Fear and Loathing over Risk Assessments Part 1

Or How We Should Be Thinking Counterfactually About Actuarial Tools

Nate Silver’s widely heralded FiveThirtyEight.com site now tracks more than just presidential elections. He and his colleagues apply statistical modeling or reasoning to everything from the Emmys to ERAs. Just over a year ago, its contributors–in collaboration with the Marshall Project (which itself is funded by the A2J Lab’s sponsor, the Arnold Foundation)–released a feature on the use of pretrial and sentencing risk assessments. So, too, did investigative journalists at ProPublica.

Both pieces raised serious questions about the use of risk scoring mechanisms. Should officials base decisions about individual arrestees or convicted defendants on aggregate data from other cases? Is there any evidence that these tools are racially biased? Former Attorney General Eric Holder previously voiced those concerns, best captured in his statement: “Although these measures were crafted with the best of intentions, I am concerned that they inadvertently undermine our efforts to ensure individualized and equal justice. . . . [T]hey may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society.” […]

Why RCTs? Part 3

Why Randomized Controlled Trials (RCTs)?

Pt. 3: Scared Straight

This is Part 3 in a series called “Why RCTs?” which explores experiences with and without the benefit of randomized study across disciplines. You’ve read about robot babies and hormone replacement therapy.  We now bring you an example from the criminal justice system that will show you the possible consequences of implementing programs without knowledge of their effectiveness- even when we have a good gut feeling of how those programs should work.


Why RCTs? Part 2

Why Randomized Controlled Trials (RCTs)?

Pt. 2: Hormone Replacement Therapy

This is Part 2 in a series called “Why RCTs?” which explores experiences with and without the benefit of randomized study across disciplines. In case you weren’t shocked enough by the robot babies account from last week, this blog post explores another catastrophic outcome, this time in the medical field. This time, the RCT produced results drastically different from those in prior observational studies, causing the medical community to reconsider long-held beliefs about the benefits of a treatment for cancer patients.


Assessing Pretrial Risk Assessment

Of Scores and Simulations

Evidence-based practices in criminal law proceedings are rapidly gaining traction. Some high-level officials and scholars have voiced concerns about their use in sentencing, as the empirical turn in judicial decision-making continues to gain momentum. A significant portion of the A2J Lab’s mission is to take the practice community’s concerns seriously and then rigorously evaluate solutions to procedural problems.


Why RCTs? Part 1

Why Randomized Controlled Trials (RCTs)?

Pt. 1: Robot Babies

Welcome! This is the first post in a series called “Why Randomized Controlled Trials (RCTs)?” which will explore experiences across disciplines. Here we explain how RCTs have often turned commonly accepted beliefs on their heads and reveal how rigorous testing is needed even when, and perhaps especially when, we assume to know the best practices in a field.


We are the A2J Lab

Hello, World!

We are the Access to Justice Lab.  The A2J Lab is an Arnold-Foundation-funded initiative within the Harvard Law School Center on the Legal Profession.  Our mission is to produce rigorous evidence in the fields of access to justice (civil or criminal) and adjudicatory administration, and to combat the resistance within the U.S. Bench and Bar to rigorous empirical thinking.