Or How We Should Be Thinking Counterfactually About Actuarial Tools
Nate Silver’s widely heralded FiveThirtyEight.com site now tracks more than just presidential elections. He and his colleagues apply statistical modeling or reasoning to everything from the Emmys to ERAs. Just over a year ago, its contributors–in collaboration with the Marshall Project (which itself is funded by the A2J Lab’s sponsor, the Arnold Foundation)–released a feature on the use of pretrial and sentencing risk assessments. So, too, did investigative journalists at ProPublica.
Both pieces raised serious questions about the use of risk scoring mechanisms. Should officials base decisions about individual arrestees or convicted defendants on aggregate data from other cases? Is there any evidence that these tools are racially biased? Former Attorney General Eric Holder previously voiced those concerns, best captured in his statement: “Although these measures were crafted with the best of intentions, I am concerned that they inadvertently undermine our efforts to ensure individualized and equal justice. . . . [T]hey may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society.” […]