Predisposition Criminal Process

The Setting: This project will interrogate the effectiveness of a light-touch court-led diversion program that educates persistent low-level offenders about crime risk factors and connects them to community resources to address socio-economic needs as a strategy to prevent further engagement with the criminal justice system.

The Problem: Light-touch diversion programs could benefit many, including those incarcerated predisposition due to inability to meet court-imposed release conditions, such as paying money bail, finding an open slot in monitoring or treatment programs, or identifying a permanent residence. In addition, light-touch diversion programs (such as the one studied here) could help those guilty of offenses whose comparatively minor significance makes it difficult to justify (morally, fiscally, and legally) a more intrusive or punitive intervention. In both instances, any benefits would likely inure disproportionately to minority communities, the impoverished, the homeless, and those in need of specialized mental health treatment. The lightness of the touch, and thus the reduced expense, of the program makes it a likely subject of replication. But these benefits will manifest only if light-touch programs succeed in reducing recidivism and improving lives. If not, the criminal justice system must confront tough choices. Likely, the system must either explore more intrusive interventions that will be difficult to justify given the low-level nature of the triggering offenses (or the defendant’s pre-guilt status) or confront the prospect of decriminalizing certain behavior.

The Questions: The research questions are: Does light-touch court-led diversion addressing root causes of risky behavior reduce continued arrests/citations, charges, and convictions among persistent low-level offenders?  Does light-touch diversion’s connection with community-based social services improve the housing and employment stability of its participants?

The Study: To understand the effectiveness of the diversion program targeted here, the Access to Justice (A2J) Lab will conduct a randomized control trial (RCT). It will randomly assign consenting study participants to either the diversion program (the “Diversion Group”) or the typical prosecutorial criminal justice process (the “Status Quo Group”).

What We’ll Learn: We will use official/administrative records to understand program effects on criminal case outcomes, recidivism (arrests, charges, and convictions), housing, and employment stability. We will also attempt to investigate whether the diversion program’s connection to community service organizations is enough to increase service take-up.

Research Team:
Access to Justice Lab
Toledo Municipal Court

Resources:
“Toledo Municipal Court Announces Study of Cutting-Edge Diversion Program,” Toledo Municipal Court press release
“Study of Community Diversion Program Launches,” Access to Justice blog, Harvard Law School

Justice Study Frequently Asked Questions

Research Team:
Michel Maréchal, University of Zurich
Julian Langer, University of Zurich
Andreas Beerli, ETH-Zurich
Therapeutic Justice Foundation
RISE

The Problem: Since the formation of public defender offices, defense attorneys have sometimes incorporated social workers into the criminal defense team. More recently, some social worker involvement has extended to the provision of services that go beyond the clients’ legal needs. A minority of states (about 14) currently make use of this holistic defense approach, and some are working to expand their use of social workers. Other states are considering whether to adopt a holistic defense approach that incorporates social workers. But there is no credible evidence that incorporating social workers into the criminal defense team, as opposed to advising defendants to make use of social services available in their communities, improves either criminal justice outcomes or improves defendant quality of life. There is also no evidence to suggest that lawyers allocate scarce social worker resources well.

The Questions: Does integrating social work services into the public defense team reduce recidivism and improve other outcomes for defendants? How well do public defenders determine which defendants would most be helped by a social worker?

The Study: Working with Sarah Buchanan, Director of Social Services at the Knox County Public Defender’s Community Law Office, the A2J Lab has designed a four-site, double-randomized evaluation to be implemented in multiple jurisdictions in Tennessee. In all four sites (Jackson, Franklin, Kingston, and Knox Counties), the RCT will investigate whether outcomes with a social worker as part of the criminal defense team are different from those where defendants are invited to take advantage of social workers in their communities. In Knox County, after attorneys make a provisional decision regarding incorporation of a social worker into the criminal defense team, the A2J Lab will randomize each potential client into one of two groups (the first level of randomization). In the first group, the attorney’s decision will be followed; in the second group, a random decision will replace the attorney’s. Among that second group, a computer-based randomizer will “decide” whether the defendant will be offered a social worker as part of the criminal defense team. In the other three counties, a randomizer will determine whether the defendant is offered a social worker as part of the criminal defense team; these three counties will not evaluate attorney triage decisions. In all the sites, the study team will follow various outcomes for each participant, including recidivism, for two years after randomization.

What We’ll Learn: The RCT design will discover how effective incorporating a social worker into the criminal defense team is at decreasing recidivism as well as housing insecurity, unemployment, and other risk factors for future criminal behavior. The double randomization in Knox County will also determine how successful lawyers are at determining which clients would most benefit from the scarce social worker resources.

Research Team: 
Sarah Buchanan (formerly of TN Pub Def Office)
April Faith-Slaker, Executive Director, Texas Access to Justice Commission

The Setting: The Public Safety Assessment-Decision Making Framework (PSA-DMF) System is a risk assessment instrument and decision-making framework designed to provide an accurate tool to distinguish criminal defendants according to the risk that they will engage in future misconduct. Jurisdictions across the United States have begun to implement the PSA-DMF System at the initial release/detention hearing at the start of a criminal case.

The A2J Lab will evaluate the effects of the Public Safety Assessment-Decision Making Framework (PSA-DMF) System report in several jurisdictions across the country. The purpose of the study is to evaluate the impact of the report, which eliminates the need for arrestee interviews, on several factors, including new criminal activity, the number of days individuals spend incarcerated pretrial; and racial and gender fairness. (Visit the Open Science Framework page about this study for more information.)

The A2J Lab is also completing four validation studies of the PSA. Unlike most of the A2J Lab’s studies, these studies are not RCTs. A validation study uses statistical analysis to determine how well the PSA classifies what it was designed to classify, i.e., the risk that certain outcomes will occur.

What We’ll Learn: The A2J Lab is conducting randomized control trials to assess whether providing the PSA-DMF System report to judicial officials making initial release/detention decisions leads to reductions in the following outcomes:

• failures to appear at future hearings
• new criminal activity
• new violent criminal activity
• the number of days defendants spend incarcerated pretrial

Research Team: 
Chris Griffin, James E. Rogers College of Law, University of Arizona
Matthew Stubenberg, Innovator in Residence, William S. Richardson School of Law at the University of Hawaii

The Setting: Numerical risk assessments feature an enormous variety of settings, including medicine, genetics, project management, environmental engineering, human resources, and law. In the legal system, risk assessments are in use at arrest, bail, charge selection, sentencing, probation, prison assignment, and parole stages of the criminal process.

Numerical risk assessments are scores usually transformed into bands (e.g., low, medium, high) that purport to provide a quantitative measurement of an individual’s or a situation’s risk that an undesired outcome will occur unless a decision maker intervenes in some way. This set of studies concerned a risk assessment instrument purporting to guide a judge’s decision on whether, and under what circumstances, to release a recently arrested individual who awaits disposition of charges against them, and if so, whether the individual should achieve release only if they first post bail or bond and only if they comply while free with monitoring conditions.

Arnold Ventures’ Public Safety Assessment-Decision Making Framework (PSA-DMF) System is a risk assessment instrument and corresponding decision-making guidance system. The PSA takes as its inputs eight criminal history variables and one demographic variable, age. It produces two 1-6 scores purportedly assessing an arrestee’s risk of failure to appear (FTA) and rearrest (new criminal activity, or NCA) if released without bail or monitoring conditions. The PSA also includes a 0-1 new violent criminal activity (NVCA) flag, with a 1 purportedly signifying elevated NVCA risk. The DMF is jurisdiction-specific, and it combines the PSA scores with matters of local concern, such as state law, community risk tolerance, and community resources to produce a recommended decision for the judge. Jurisdictions across the United States provide PSA-DMF system printouts to judges deciding predisposition release, bail, and monitoring conditions.

Risk assessment instruments, despite their promise, have given rise to concerns worthy of study. Some have argued that they create, further, or calcify racial bias. Others contend that the instruments rely on subjective factors, which require the exercise of judgment to score; that their classifying power has not been adequately assessed; that most instruments depend on information that, realistically, can only be obtained from an arrestee interview (the PSA does not have this problem); and, crucially, that only one instrument has undergone randomized evaluation, with that evaluation occurring more than six decades ago.

The Problem: Despite their ubiquity, risk assessment instruments have rarely been the subject of credible study, and the only RCT concerning them occurred over six decades ago. Thus, no one knows whether use of predisposition risk assessments reduces some or all of FTA, NCA, NVCA, number of days of predisposition incarceration, and measures of racial fairness.

The Questions: Do risk assessment instruments allow judges to make better decisions regarding assignment of bail and monitoring conditions, thereby reducing some or all of FTA, NCA, NVCA, number of days of predisposition incarceration, and measures of racial fairness?

The Studies: After several years of planning, the A2J Lab in 2017 launched a series of RCTs of the PSA-DMF System in Dane County, WI; in Polk County, IA; and in four Utah counties: Davis, Utah, Weber, and Morgan. In Dane and Polk, each first appearance hearing was randomized either to a PSA-DMF-available condition or to a PDA-DMF-not-available condition. In the former, the deciding judge had access to a PSA-DMF printout containing the PSA scores, the DMF recommendation, and the underlying PSA inputs. In the latter, the PSA-DMF printout was produced but was not provided to the deciding judge. In the four Utah counties, the process was slightly different because there was no live hearing, so the deciding judges either did or did not have access to the PSA-DMF (electronic) “printout” when they adjudicated bail and monitoring conditions (a task they accomplished as they simultaneously decided whether probable cause to arrest was present). In addition, in the Utah counties, randomization was by unique individual person, not by arrest, so a person with multiple arrests over the lifetime of the study was not re-randomized.

What We Learned And What We’ll Learn: Thus far, only interim results and some partial final results are available from the Dane County study. Those results suggest that despite changes in judges’ decision making, changes that conformed to the PSA-DMF system recommendations, the available ability of the PSA-DMF System printout produced no statistically significant alterations in FTA, NCA, NVCA, number of days of predisposition incarceration, or measures of racial fairness. Preliminary analysis, not yet public, suggests that the same was true in Polk, despite even larger shifts in judicial decision making in accordance with PSA-DMF recommendations. If these results hold up, they will demonstrate that at least this risk assessment instrument produces neither the benefits nor the harms that its proponents and detractors predicted. Such a result would demand research into why no effects emerged. Possible reasons for the lack of results include (i) that the PSA did not classify arrestee risk well enough to make a difference in FTA, NCA, etc., and (ii) that the intervention that judges use to try to affect these outcomes, namely imposition of bail and monitoring conditions, are themselves ineffective, meaning that even if the PSA accurately measures risk, there is no effective way to change that risk.

In addition, this research program has already, and will continue to, produce advances in statistical methodology (see Resources below).

Research Team:
D. James Greiner, Faculty Director, Access to Justice Lab; S. William Green Professor of Public Law, Harvard Law School
Renee L. Danser, Associate Director of Research and Strategic Partnerships, Access to Justice Lab
Chris Griffin, Director of Empirical & Policy Research; Research Professor, James E. Rogers College of Law, University of Arizona
Matthew Stubenberg, Innovator in Residence, William S. Richardson School of Law at the University of Hawaii
Ryan Halen, Data Analyst, Access to Justice Lab
Heidi Liu, J.D./Ph.D. candidate, Harvard Law School and Harvard Kennedy School
Kosuke Imai, Professor, Department of Government and Department of Statistics, Harvard University; Affiliate, Institute for Quantitative Social Science, Harvard University
Zhichao Jiang, Professor, School of Mathematics, Sun Yat-sen University
Sooahn Shin, PhD candidate, Department of Government, Harvard University; Affiliate, Institute for Quantitative Social Science, Harvard University
Eli Ben-Michael, Assistant Professor, Department of Statistics & Data Science, Heinz College of Information Systems and Public Policy, Carnegie Mellon University

Resources:
Kosuke Imai, Zhichao Jiang, D. James Greiner, Ryan Halen, Sooahn Shin, “Experimental Evaluation of Algorithm-Assisted Human Decision-Making: Application to Pretrial Public Safety Assessment*,” Journal of the Royal Statistical Society Series A: Statistics in Society, 2023

Zhichao Jiang, Eli Ben-Michael, D. James Greiner, Ryan Halen, Kosuke Imai, “Longitudinal Causal Inference with Selective Eligibility,” arXiv, Mar. 15, 2025

Eli Ben-Michael, D. James Greiner, Kosuke Imai, Zhichao Jiang, “Safe Policy Learning through Extrapolation: Application to Pre-trial Risk Assessment,” arXiv, Mar. 31, 2025

The Setting:

What We Hope to Learn:

Research Team:
Access to Justice Lab
Raphah Institute