Sandy North

We’re closing out 2020 with a bang: We have three new studies in the field. Our amazing partners have worked with us to prepare and launch these projects in the face of the unprecedented challenges posed by COVID-19. Focusing on different topics and in different geographies, these new studies have the potential to improve access to justice across the U.S.

---

Transformative Justice Program (Williamson County, TX)

The Problem

In Texas, young adults aged 17 – 24 are overrepresented in the adult criminal justice system, accounting for 29% of the state’s arrests while only making up 11% of the population. Emerging adults also have the highest short-term recidivism of any age group due to underlying factors that are not addressed by the current criminal justice system. These factors include, among others, mental health struggles, substance abuse, and co-occurring disorders.

The Program

The program is Williamson County’s first felony diversion program. It diverts emerging adults charged with a low-level felony offense from the traditional criminal justice system. The program is a court-based system that combines release into the community with developmentally appropriate, intensive, community-based services. County staff work with participants to create individualized plans to address health, housing, educational, and other needs.

Eligible participants can be part of the program for up to 18 months, and they will be connected with community social services providers to help them meet their individualized program goals. Participants who successfully meet the goals identified in their individual plan will graduate from the program and will be eligible for expungement of the record of the arrest and charges.

The Study

In partnership with Williamson County, the Public Policy Research Institute at Texas A&M University, and the University of Texas Health Science Center at Houston (UTHealth) School of Public Health, the A2J Lab has launched a randomized evaluation of the diversion program.

The research team will conduct an RCT to identify the effects of the program on several outcomes including recidivism rates and quality of health outcomes. The study will also supplement criminal justice data with quarterly surveys and other data sources.

What We’ll Learn

This study will provide important evidence about the impact of community-based services. The population has a high level of need. Because they are so young, any benefit they experience could improve their lives and their communities for decades to come. With this research, we will learn the extent of this potential improvement.

---

Online Dispute Resolution (Carroll County, IA and Miami-Dade County, FL)

The Problem

Miami-Dade will likely issue tens of thousands of traffic compliance tickets in 2020. These are minor infractions that often simply require a motorist to show proof of license, registration, or insurance to resolve the case. The complicating factor is that the motorist needs to get to the courthouse on the right day and time and wait until it is their turn to give this proof. Similarly, the three different law enforcement agencies that police the Carroll County roadways issued thousands of traffic tickets last year. All of these tickets require an in-person appearance if there’s a discrepancy to resolve.

Attendance can be difficult, and the consequences for not being in the right place at the right time can be expensive. While the process of having these tickets resolved is often inexpensive, leaving the issue unattended can result in increased fines and fees. The courts’ solution to this?  Put it online.

The Program

With the installation of Online Dispute Resolution (ODR), motorists can now avoid the hassle of getting to the courthouse–which comes with all of the typical inconveniences of parking, reliable transportation, taking time off of work, finding childcare–and upload all of the compliance documents or negotiate discrepancies online. Prosecutors and court staff can then review what is submitted and handle the ticket without ever causing the motorist to set foot in the courtroom.

The Study

The A2J Lab is running RCTs in both counties. The research team will measure outcomes such as perceptions of the justice system, time to disposition, settlement success, and failure to appear.

What We’ll Learn

This change is happening at a critical time for state courts in public perception. As courts are moving practices online to cope with the COVID-19 pandemic, they have very little information about how those changes impact people’s perceptions of court experiences and other outcomes. This study will collect data on how using ODR impacts users’ overall perception of the justice system. This data, along with information about how cases resolve, will create evidence about how ODR improves the experiences of people in Miami-Dade and Carroll counties.

---

Plain Language Court Forms (DuPage County, IL)

The Problem

Reducing technical jargon and “legal-ese” is of major interest to access for justice advocates. For pro se litigants (people without lawyers), forms can be so complex that they require large sets of self-help materials to understand. Changing the forms themselves to be easy-to-read documents is a more efficient way to assist people without attorneys file the paperwork they need.

The Program

The Illinois Supreme Court Commission on Access to Justice created a Standardized Forms Committee to explore the creation of pro se forms that implement standardization and plain language practices. The new forms attempt to provide better guidance and support to pro se litigants by integrating plain language and simplification.

The study will include individuals who attempt to download a pro se divorce form for filing in DuPage County, Illinois. Any attempt to download a relevant form through the various websites, such as the DuPage County website (www.dupageco.org), the computers in the self-help center at the courthouse, or local legal aid organization websites will redirect to a study-based webpage. This webpage will verify eligibility for the study, provide information about the study, and subsequently provide either the new standardized pro se form or the previously created DuPage-specific pro se divorce form. These forms will be provided at random.

The Study

Through metadata and time-stamp information attached to the downloaded forms, as well as information documented in the court’s case management system, the A2J Lab will track a number of outcomes. These outcomes will include success of filing the form, time to filing, time to disposition, number of procedural errors, and whether the study participants use the downloaded form or end up reaching out to legal aid for assistance. The A2J Lab expects to conduct the field randomization for approximately six months and will track each case through disposition.

What We’ll Learn

This evaluation will give concrete evidence about what if any impact access to plain language forms has. Increasing the body of evidence about its efficacy will help courts decide how to deploy their resources.

---

Preliminary data for all of these studies will be available in the next year.

Today, the A2J Lab presented the findings from the interim report of the randomized evaluation of the Public Safety Assessment–Decision Making Framework System (PSA-DMF), a pretrial risk assessment tool and related decision-making framework, in Dane County, Wisconsin. The full interim report is now available on the A2J Lab website.

The presentation at the Dane County Criminal Justice Council was the first result of a years-long study, which is not yet complete. In the late spring of 2017, Dane County started providing the PSA-DMF System information to judicial officers deciding how much and what kind of bail and supervision to assign to individuals who have been arrested; such decisions affect whether the individual will be released or remain in jail until trial. Working with the A2J Lab and Arnold Ventures, Dane County’s randomized field experiment began a month later.

The PSA-DMF System is one tool in the toolbox that the judicial officer can draw upon in the exercise of their professional discretion. Scientists supported by Arnold Ventures produced the PSA by reviewing past data on criminal history, demographics, new crimes, rates at which participants failed to appear at their court hearing, and other potential risk factors. The PSA scores are applied to the Decision Making Framework, which brings together the information from the PSA with a community’s local policies and values, its laws, and its resources to provide a recommendation regarding pretrial release and, if release is obtained, a supervision level. Judicial officers may use the PSA-DMF System report when deciding whether to release an individual before trial, and this decision rests always with the judicial officer.

The experiment used something like a coin flip to divide cases into two groups. In the treated group, the judicial officer who was deciding how much and what kind of bail and pretrial supervision to assign received a paper printout with the PSA-DMF System information on it. In the control group, the judicial officer did not receive the paper printout. In other words, the control group received standard practice, and the treated group received the new system in which the judicial officer got the PSA-DMF System printout.

The report released this week analyzes data from one year of follow-up for cases that were included in the study between the start of the study and the middle of 2018. Dane County randomized cases from the middle of 2017 until the end of 2019. In this interim report, the A2J Lab analyzed data the County provided to compare the treated and control groups on (among other things) measurements of racial fairness, number of days incarcerated (if any), rates at which participants failed to appear at their court hearing, new criminal activity, and new violent criminal activity. Criminal justice officials in Dane County worked to provide information to the A2J Lab to facilitate the A2J Lab’s independent evaluation in a spirit of learning and a desire to improve.

Quantitatively, the A2J Lab does not yet have enough cases, or enough of a follow-up period on those cases, to make firm conclusions about whether it is better or worse to make the PSA-DMF System printout available to the judicial officer before assignment of bail (if any) and release conditions. The A2J Lab’s studies, like all randomized control trials, require a number of participants sufficient to detect policy-relevant differences between treatment groups (here, the PSA-DMF System report versus business as usual). The number of participants must be sufficient to analyze for statistical significance. Because of the two-year follow-up period, the County will provide the A2J Lab full information on all arrestees in the study sometime in early 2022. If the data are provided then, the A2J Lab’s final report will be available in the summer of 2022.

The limited data available thus far, not enough to draw firm conclusions, suggested several findings:

• There is some evidence that providing the PSA-DMF System printout to the judicial officer caused a change in the officer’s decisions.
• Generally, when the printout indicated that an individual presented lower risk, the judicial officer was less likely to require cash bail or, if cash bail was required, the amount was lower than in comparable control group cases. The opposite was generally true in the treated group when the printout indicated that the individual presented higher risk, as compared to the control group. This change was statistically significant but mild, and we cannot yet tell whether the change is policy-relevant.
• Treated group cases varied less in bail types and amounts than did control group cases. This change was strong and statistically significant.

As of the time of the interim report, there was no statistically significant difference between treated versus control group cases with respect to:

• various measures of the racial fairness of the judicial officer’s decisions;
• the number of days (if any) of pretrial incarceration;
• the frequency with which arrestees failed to appear at court dates; or
• the frequency with which arrestees were arrested for new crimes, including new violent crimes, during the pretrial period.

These finding might change, however, when the A2J Lab finishes analysis of the final dataset.

While it is too early to draw conclusions about whether the PSA-DMF System is positive, negative, or neutral for Dane County, this data provides a first look at the type of report that the final dataset will support.

Texas A&M University’s Public Policy Research Institute (PPRI) and the Access to Justice Lab at Harvard Law School (A2J Lab) are launching a randomized study of counsel at first appearance (CAFA) in Texas—the first study of its kind.

At first appearance, magistrates decide whether a person will be released while awaiting trial or held in jail. Yet only 4 of 254 Texas counties provide lawyers at this important stage. This week, the research team launches the evaluation in Hays County. Previously, counsel was typically assigned after first appearance. Going forward, the Texas Indigent Defense Commission (TIDC) will provide the funding for counsel to be present at magistration some of the time to evaluate the effect of provision of counsel.  The study will randomize days on which counsel is present for a first appearance. Eligible study participants will participate in brief interviews with study attorneys before being represented.

This study will use a randomized control trial (RCT) to test whether CAFA:

• Lowers money bond amounts
• Increases the likelihood of non-monetary bonds (personal bonds)
• Increases pretrial release
• Affects the likelihood of defendants returning to court
• Reduces jail costs and overall system costs

The Hays County COVID-19 response means that these appearances are now occurring remotely. This adjustment will allow researchers to observe a change in outcomes when appearance is remote versus when appearance is in-person upon resumption of normal operations. This observational analysis may pave the way to understanding the value of online courts.

Hays County Precinct 3 Commissioner Lon Shell said, “Hays County is excited to participate in this study. We believe it will provide vital information that can be used to continue improvements to our justice system.”

George Naufal, associate research scientist at the PPRI at Texas A&M University, says, “Understanding the impact of appearance at this stage will provide a huge benefit to the people of Texas. As more jurisdictions consider adding counsel at this stage, providing them with information about the effectiveness will help them decide if it’s a prudent use of their resources.”.

Jim Greiner, Faculty Director of the Access to Justice Lab at Harvard Law School, says, “The decisions made at the first court appearance have a huge impact. Those decisions mean the difference between spending days, weeks, months, or, in some cases, years in jail before a trial even occurs. With this study, we’ll get concrete evidence about whether providing counsel at those appearances makes a meaningful difference. If it does, this program could become a model for others.”

The study is supported by Arnold Ventures and the Texas Indigent Defense Commission.

Are you interested in learning about managing RCTs in the legal profession?

Work with us—we’re hiring!

We have an opening for a new Research Specialist to support the operations of some of our current research studies. If you’re interested in hands-on experience of quantitative research in law, we’d love to hear from you.

All inquires must be submitted through Harvard Law School’s online application portal.

We’re currently accepting applications. If you’re interested, please apply as soon as possible.

In partnership with Alaska Legal Services Corporation (ALSC), the A2J Lab has launched a randomized control trial to evaluate community advocates’ services in the context of the Supplemental Nutrition Assistance Program (SNAP – aka Food Stamps) benefits arm of their programs.

Low-income and elderly residents in Alaska are in need of access to these benefits. Food insecurity and hunger are especially threatening to Alaska’s rural regions. Currently, there are 1.3 ALSC lawyers for every 10,000 Alaskans in poverty; and, the majority of the state’s 2,364 active attorneys are concentrated in the third judicial district, leaving much of its rural population without representation.

In response to this need, ALSC launched its community advocates program. The program will increase the pool of non-attorney volunteers, such as paralegals, tribal legal advocates, and professional school and law school students who reside in various communities throughout the state. Through training materials and supportive services these advocates will be able to provide valuable services in communities where there are currently no or very few attorneys.

To study the effectiveness of this program, ALSC will process approximately 300 study-eligible individuals, half of whom will be assigned a lawyer, the other half assigned a trained community advocate to provide a range of support services. Community advocates will be located throughout the state and began providing assistance to clients last year. The A2J Lab will compare the control group results and establish training programs to enhance resources already in place.

If the project proves effective, the benefits across a variety of civil justice needs in Alaska will be enormous. Key stakeholders will be enabled to implement the best solutions for their regions. Residents will gain access to the federal benefits they deserve.

In the time of COVID-19, ensuring that everyone who qualifies for SNAP benefits gets them is crucial. Because the study happens without additional in-person contact, we’re able to continue the research to learn more about how this system works at a time when people need the services more than ever.

The results will echo beyond the state. This project is one of several that the A2J Lab has in development to evaluate non-lawyer services. With the demand for civil legal aid far outstripping supply across the country, understanding if and when non-lawyer assistance might help, could improve results for clients who otherwise would not be able to access any support at all.