• Skip to primary navigation
  • Skip to main content
Access to Justice Lab logo

The Access to Justice Lab

at Harvard Law School

  • About
  • Projects
  • Publications
  • Resources
  • People
    • A2J Lab Staff
    • Advisory Board
    • Collaborators
  • Contact
  • Donate
  • Show Search
Hide Search

rct

The TN Social Work in Public Defense Study Has Launched!

December 21, 2018 by Sandy North

April Faith-Slaker has been on the road this past week launching the Lab’s newest field operation. Working closely with a group of collaborators throughout Tennessee, April has over the past eight months co-developed and now launched an evaluation of the impact of a holistic defense approach that includes social workers on the public defense team.

Support for a holistic approach to public defense has been growing across the United States[1], grounded in the recognition that indigent clients have a wide range of pressing needs that go beyond the legal case. The types of problems that many public defense clients face – such as drug addiction, poverty, and homelessness – cannot be adequately addressed by a lawyer alone. Holistic defense implements a multidisciplinary approach to improve client outcomes and reduce recidivism by addressing contributing factors that keep people in or cycling through the criminal justice system.

Pictured left to right: Sarah Buchanan, Director of Social Services and Isaac Merkle, Director of Information Technology

The use of social workers has been integral to this approach, yet little to no rigorous studies have examined the effectiveness of social workers in this setting. Based on initial research completed by one of the study collaborators, Sarah Buchanan, Director of Social Services at the Knox County Public Defender’s Community Law Office,  the study design tests whether or not integrating social work services into the public defense team reduces recidivism and improves other outcomes for clients. April has been working closely with Sarah Buchanan to pick up where her initial research left off, by implementing a randomized control trial study of social work in public defense. In a recent correspondence between the A2J Lab and Buchanan, Buchanan stated:

“Social workers have been working toward the development of effective and socially just alternatives to incarceration for vulnerable populations in public defender systems across the country for many years without much guidance. Measuring the impact of social work services is critical as the social work profession continues to integrate into public defense settings and work toward addressing issues such as smart decarceration, economic inequality, equal opportunity and justice.”

In this RCT study, participants will be randomly assigned to one of two conditions. In the treated group, the clients will be assigned a social worker from the public defender office. The social worker will work directly with the lawyers and the clients to provide or arrange for social services. In the control group, the clients will receive a pamphlet listing community social workers and services who provide equivalent services. Over time, the research team will track outcomes, including recidivism, to learn what impact the treatment has.

The randomized methodology will take place over the next year in four public defender offices in Tennessee – Knoxville, Franklin, Jackson, and Kingston. The latter three offices all integrated social workers in July 2018. Knoxville – the largest site – has had over a decade of experience with this holistic defense model, and their attorneys are extremely attuned to this client-centered approach. Therefore, in this site, the study will not only examine the effectiveness of the social workers, but will also examine the way in which the lawyers in the office make triage decisions regarding which clients receive a CLO social worker.

This study has the potential to provide significant information about the impact of social work services. The integration of social workers in this setting is growing, but by no means the norm in public defender offices. Most states have districts or offices that have added social workers, but at this time it is mostly on a limited scale. As other jurisdictions consider whether to adopt a holistic defense approach that incorporates social workers or broaden their programs, having empirical evidence about the impact of this program could help those jurisdictions make more informed decisions about how to design their programs.

The A2J Lab is very grateful to the whole research coalition for their participation in this landmark study. This type of project is a true team effort; we couldn’t do it without the hard work and commitment of everyone involved.

[1] See Steinberg, R. (2013). Heeding Gideon’s call in the twenty-first century: Holistic defense and the new public defense paradigm. Washington & Lee Law Review, 70, 961-1018

Previewing and Reviewing Pretrial Risk Assessment RCTs

January 18, 2018 by Chris Griffin

On Tuesday, Jan. 16 pretrial staff in Polk County, Iowa entered their offices with a slightly different charge. They had been accustomed to perusing a list of arrestees scheduled for first appearance and searching for individuals who qualified for an interview and pre-disposition release. That morning, some staff members continued this time- and resource-intensive practice. Others reviewed administrative records and entered nine risk factors into a new software system that calculates (hopefully familiar to readers of this blog) PSA risk scores. Polk County is the first jurisdiction in Iowa to implement the PSA. Three more counties will join them in the coming months as pilot sites, and eventually the entire state will adopt it.

As the A2J Lab looks ahead to launching its second RCT evaluation of the PSA, we came across a study of its progenitor, the Virginia Pretrial Risk Assessment Instrument (“VPRAI”). When the VPRAI arrived in courtrooms around the state, there was no way to convert risk predictions into actionable release recommendations. (That fact stands in stark contrast to the Decision-Making Framework accompanying the PSA.) The solution was the Praxis, “a decision grid that uses the VPRAI risk level and the charge category to determine the appropriate release type and level of supervision.” Virginia pretrial staff also embraced the so-called Strategies for Effective Pretrial Supervision (“STEPS”) program to “shift the focus . . . from conditions compliance to criminogenic needs and eliciting prosocial behavior.” The combination of these innovations, it seemed, would improve Virginia’s ability to pinpoint risk and reduce failure rates during the pre-disposition period.

Marie VanNostrand of Luminosity and two co-authors were interested in understanding, first, the VPRAI’s predictive value. Second, they assessed the benefits of the Praxis and STEPS program through a randomized study design. Unlike the A2J Lab’s field experiments, which usually take individuals as the units of randomization, the Virginia study randomized entire pretrial services offices to one of four conditions: (1) VPRAI only; (2) VPRAI + Praxis; (3) VPRAI + STEPS; and (4) VPRAI + Praxis + STEPS. The authors then used this exogenous (nerd speak for “completely external”) source of variation to analyze staff, judicial, and defendant responses.

The results were quite favorable for the introduction of the Praxis as well as for the VPRAI itself. One estimate suggested that higher VPRAI risk scores correlate strongly with higher actual risk. About two-thirds of the time, if one were to pick two defendants at random–one who failed and one who didn’t–the one who failed would have a higher VPRAI score. Pretrial services staff who had access to the Praxis also responded to its recommendations. Their concurrence (agreement) rate was 80%, and they were over twice as likely to recommend release relative to staff who did not have the decision grid. Next, the availability of the Praxis (versus not having it) was associated with a doubling of the likelihood that judges would release defendants before disposition.

What about defendant outcomes? The authors found that the availability of the Praxis was associated with a lower likelihood of failing to appear or being arrested for a new crime. STEPS alone had no discernible effect.

The VPRAI study suggests a few lessons for our ongoing pretrial risk assessment work, including in Iowa. First, we continue to emphasize that the tool under investigation, the PSA, is far from a cold, lawless automaton, as many commentators seem to worry. Yes, algorithms produce scores, and decision matrices generate recommendations. But human beings must still consider that evidence alongside their own human judgment. One hope is that such evidence will enhance the quality of judges’ decision-making. For now, we just don’t know; that’s the reason for our PSA RCTs. Relatedly, we think that final verdicts on actuarial risk assessments should await reports like the VPRAI study and the A2J Lab’s growing portfolio of evaluations. There will always be local policy issues deserving of debate and attention. However, we need strong evidence for or against these tools’ value before praising or condemning them wholesale. Finally we should, as always, evaluate this brave new world reliably. That means deploying, where possible, principles of experimental design. RCTs, simply put, represent our best shot at understanding causal relationships.

Stay tuned for more updates from Iowa and beyond!

RCTs in law: the Shriver studies

December 7, 2017 by Sandy North

As you may remember from a previous post, the A2J Lab is developing an RCT in Providence, Rhode Island to study the effectiveness of triage in summary eviction cases.

Part of our interest in studying eviction is that it’s a topic very much on policymakers’ minds. Because housing instability continues to receive a lot of attention (see, e.g., Matthew Desmond’s Evicted) more resources and political action tend to follow. The even better news is that those resource allocations and policy changes have been based, at least sometimes, on empirical research.

As the Lab has documented elsewhere, access to justice interventions often aren’t studied at all. If they are, the studies are often observational rather than randomized—and readers know how important we think randomization is here at the Lab!

We’re always very excited to see the work of other legal studies teams who think so as well. One recent example is the evaluation of the California Shriver Civil Counsel Act. Among other things, the Act enhanced tenant representation in eviction cases. As part of a trial for additional funding provided by the Act, the State studied the impact of seven pilot programs designed to increase access to legal representation among low-income populations in California. More than simply moving toward empirical data, analysis in the report states that: ”[i]mportantly, for a limited period of time, three pilot projects randomly assigned litigants to receive Shriver full representation or no Shriver services, and data for these two groups were compared.”

We’re excited to see other researchers embrace randomized evaluations and policymakers appreciate their findings. The ultimate hope is that this progress continues to transform the legal profession into a more evidence-based one.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Why RCTs? Recent study on stents is one example

November 15, 2017 by Sandy North

This past week, we’ve been avidly watching reactions to a new study, published in The Lancet, about the efficacy of using stents to help patients with chest pain. The New York Times ran an article on the study; so did The Atlantic.

If you haven’t been following this (potential) bombshell of an RCT, the study found no value in using stents to combat heart pain. Why is this such big news? Partially because using stents for cardiac pain is big business. According to the study’s authors, more than 500,000 patients receive the procedure annually for chest discomfort.

It’s also big news because it goes against intuition, even the sort that medical laypeople possess. Without evidence to the contrary, it might seem logical that opening blocked arteries with a stent would reduce chest pain. No wonder doctors adopted the practice with vigor! Now there are data that don’t back up that perception. Even in medicine, a field long conditioned to accepting the validity of empirical research, studies will bump up against the fallacy of conventional wisdom.

That fact doesn’t surprise us at the A2J Lab. What did grab our attention is that the authors received permission to run the study at all. As we mentioned in a recent post, all RCTs in the U.S. need to receive institutional approval before human subjects can enroll in a study. Based on our experience, it would be fairly startling if this type of study, which flies so baldly in the face of “conventional wisdom,” were to receive approval in the United States. An ethical review committee could have responded that this evaluation would prevent some participants from receiving a “benefit,” namely the treatment they “need.” The deeper held the belief, the harder it is to accept or allow the introduction of contrary evidence. That’s why we need to test interventions rigorously, particularly when resources are scarce and lives are at stake.

One final note on the study’s design. Critiques from medical researchers have included that the study is flawed due to “Type II error.” In short, they contend that the sample size (in this case, about 200) is insufficient to rule out false negatives. The challenge of having sufficient sample size is an important component of any RCT. The Lab, for example, uses power analysis to maximize the chance that a study will have enough observations to detect an effect, should that effect really exist. But a study’s sample size isn’t the only factor that’s important in determining its validity; it’s also important to know how generalizable the results are, regardless of their statistical significance.

This is just one more example of why RCTs are important. Have you seen others recently? Share them with us in the comments or on social media.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

RITES Goes Local

October 30, 2017 by Sandy North

We’re excited to announce a new collaborative venture on the Rhode Island Triage and Eviction Study (“RITES”) team. Five outstanding students at the Roger Williams University School of Law have joined the project to conduct summary eviction court observations and interview unrepresented tenant-defendants who are experiencing the eviction process themselves. These raw data, so to speak, will give everyone working on the project a better understanding of the current system and inform ongoing study design. For example, information about what tenants would have liked to know before their hearings will enhance the self-help materials designed for the evaluation.

In addition to this unique learning experience, their participation strengthens the project’s connections within the Providence community. These dedicated students—under the supervision of Professor Jonathan Gutoff; Director of the Feinstein Center for Pro Bono & Experiential Education, Laurie Barron; and Director of Pro Bono & Community Partnerships, Eliza Vorenberg—will bring important new perspectives and added capacity to the growing RITES field organization.

This partnership is an exciting one for us at the Lab. One of our core missions is to work with the next generation of legal practitioners and scholars and cultivate dedication to making the law an evidence-based profession. Having those future leaders actively participate in all phases of a Lab study is the most effective way for us to do just that.

We also prioritize having multiple partners work together on RCTs to maximize their impact. Having RWUSOL join us and the Rhode Island Center for Justice as we study the latter’s triage process for assigning representation in eviction cases will only improve our work. We all will benefit from each other’s perspectives, ideas, and insights into the understudied process of distributing scarce legal resources.

Stay tuned for more on this venture!

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

  • Go to page 1
  • Go to page 2
  • Go to Next Page »

Using empirical research to make the U.S. justice system work better for everyone.

The Access to Justice Lab

Copyright © 2023 · Monochrome Pro on Genesis Framework · WordPress · Log in

 

Loading Comments...