These instruments, despite their promise, give rise to some concerns worthy of study. Many rely on subjective factors, which require the exercise of judgment to score; their predictive power has not been assessed using data from the jurisdictions in which they are used; most instruments depend on information that, realistically, can only be obtained from an arrestee interview; and, crucially, only two instruments have undergone randomized evaluation in over six decades of use in the United States.
The Studies
The A2J Lab will evaluate the effects of the Public Safety Assessment-Decision Making Framework (PSA-DMF) System report in several jurisdictions across the country. The purpose of the study is to evaluate the impact of the report, which eliminates the need for arrestee interviews, on several factors, including new criminal activity, the number of days individuals spend incarcerated pretrial; and racial and gender fairness. (Visit the Open Science Framework page about this study for more information.)
The A2J Lab is also completing four validation studies of the PSA. Unlike most of the A2J Lab’s studies, these studies are not RCTs. A validation study uses statistical analysis to determine how well the PSA classifies what it was designed to classify, i.e., the risk that certain outcomes will occur.
The PSA-DMF System
The Public Safety Assessment (“PSA”) uses nine factors to generate scores that assess risk of three outcomes—failure to appear pretrial, new criminal arrest while on pretrial release, and new violent criminal arrest while on pretrial release. Judicial officers use the PSA scores along with a Release Conditions Matrix also known as a Decision Making Framework (“DMF”) to inform pretrial release decisions. To learn more about the factors please visit https://advancingpretrial.org/psa/factors/. The report generated by the PSA and DMF is called the PSA-DMF System report.
Field Operation: Dane County, Wisconsin
The A2J Lab is currently evaluating the effects of the PSA-DMF System report in Dane County, Wisconsin.
In Dane County, cases were randomized either to the “treatment” group, in which instance the Clerk’s Office produced the PSA-DMF System report printout, appended it to the case file, and made it available to the prosecuting and defense attorneys as well as to the Commissioner in time for the initial appearance; or to the “control” group, in which instance, the Clerk’s Office did not produce the PSA-DMF System report printout.
The study randomized and enrolled participants starting in May 2017 and enrollment concluded on December 31st, 2019. The A2J Lab will observe participant’s contacts with the court for two years following the initial appearance. Data observation will be concluded on January 1st, 2022.
The A2J Lab has drafted an interim report (and a FAQ sheet about the report) based on the first year of randomization combined with an observation period of one year for each defendant. The results are not conclusive as additional data is required which will not be available until 2022.
Outcomes
The principal outcomes of interest for this study include the following:
- Failures to appear at future hearings;
- New criminal activity;
- New violent criminal activity;
- The number of days individuals spend incarcerated pretrial; and
- Racial and gender fairness
The Research Team
Jim Greiner, Faculty Director, Access to Justice Lab; Professor of Law, Harvard Law School
Matthew Stubenberg, Associate Director of Legal Technology, Access to Justice Lab
Ryan Halen, Data Analyst, Access to Justice Lab
Heidi Liu, J.D./Ph.D. candidate, Harvard Law School and Harvard Kennedy School
Chris Griffin, Director of Empirical & Policy Research; Research Professor, James E. Rogers College of Law, University of Arizona