Why RCTs? Recent study on stents is one example

This past week, we’ve been avidly watching reactions to a new study, published in The Lancet, about the efficacy of using stents to help patients with chest pain. The New York Times ran an article on the study; so did The Atlantic.

If you haven’t been following this (potential) bombshell of an RCT, the study found no value in using stents to combat heart pain. Why is this such big news? Partially because using stents for cardiac pain is big business. According to the study’s authors, more than 500,000 patients receive the procedure annually for chest discomfort.

It’s also big news because it goes against intuition, even the sort that medical laypeople possess. Without evidence to the contrary, it might seem logical that opening blocked arteries with a stent would reduce chest pain. No wonder doctors adopted the practice with vigor! Now there are data that don’t back up that perception. Even in medicine, a field long conditioned to accepting the validity of empirical research, studies will bump up against the fallacy of conventional wisdom.

That fact doesn’t surprise us at the A2J Lab. What did grab our attention is that the authors received permission to run the study at all. As we mentioned in a recent post, all RCTs in the U.S. need to receive institutional approval before human subjects can enroll in a study. Based on our experience, it would be fairly startling if this type of study, which flies so baldly in the face of “conventional wisdom,” were to receive approval in the United States. An ethical review committee could have responded that this evaluation would prevent some participants from receiving a “benefit,” namely the treatment they “need.” The deeper held the belief, the harder it is to accept or allow the introduction of contrary evidence. That’s why we need to test interventions rigorously, particularly when resources are scarce and lives are at stake.

One final note on the study’s design. Critiques from medical researchers have included that the study is flawed due to “Type II error.” In short, they contend that the sample size (in this case, about 200) is insufficient to rule out false negatives. The challenge of having sufficient sample size is an important component of any RCT. The Lab, for example, uses power analysis to maximize the chance that a study will have enough observations to detect an effect, should that effect really exist. But a study’s sample size isn’t the only factor that’s important in determining its validity; it’s also important to know how generalizable the results are, regardless of their statistical significance.

This is just one more example of why RCTs are important. Have you seen others recently? Share them with us in the comments or on social media.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Top 10 A2J Research Priorities #10: Ex Ante Law for Human Beings

We’ve reached the final installment in our Top 10 A2J Research Priorities series. While this is the last of the videos, it’s just the beginning of our work on these topics.

This week’s segment features Faculty Director Jim Greiner on Ex Ante Law for Human Beings.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

RITES Goes Local

We’re excited to announce a new collaborative venture on the Rhode Island Triage and Eviction Study (“RITES”) team. Five outstanding students at the Roger Williams University School of Law have joined the project to conduct summary eviction court observations and interview unrepresented tenant-defendants who are experiencing the eviction process themselves. These raw data, so to speak, will give everyone working on the project a better understanding of the current system and inform ongoing study design. For example, information about what tenants would have liked to know before their hearings will enhance the self-help materials designed for the evaluation.

In addition to this unique learning experience, their participation strengthens the project’s connections within the Providence community. These dedicated students—under the supervision of Professor Jonathan Gutoff; Director of the Feinstein Center for Pro Bono & Experiential Education, Laurie Bannon; and Director of Pro Bono & Community PartnershipsEliza Vorenberg—will bring important new perspectives and added capacity to the growing RITES field organization.

This partnership is an exciting one for us at the Lab. One of our core missions is to work with the next generation of legal practitioners and scholars and cultivate dedication to making the law an evidence-based profession. Having those future leaders actively participate in all phases of a Lab study is the most effective way for us to do just that.

We also prioritize having multiple partners work together on RCTs to maximize their impact. Having RWUSOL join us and the Rhode Island Center for Justice as we study the latter’s triage process for assigning representation in eviction cases will only improve our work. We all will benefit from each other’s perspectives, ideas, and insights into the understudied process of distributing scarce legal resources.

Stay tuned for more on this venture!

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

MA Guardianship Policy Institute Event this December

The Lab’s current Guardianship Service of Process Study is going strong in the field, and we’re not the only ones actively thinking about the topic. On Wednesday, Dec. 6, the Massachusetts Guardianship Policy Institute will facilitate “A National Perspective on Guardianship and Decisional Support,” a day-long conference exploring the state of both areas in several different contexts. You can learn more about the conference here.

This event will present several different perspectives, and the Lab is always looking for new lenses through which to view fields we study. The A2J Lab doesn’t have an opinion about whether guardianship, or alternatives to it, are good or bad, nor about the situations in which any alternatives are appropriate. We believe that as long as guardianship is a step that the law makes available, its availability should turn on the criteria outlined in governing law and not (as it appears to now) on whether a petitioner has a lawyer or is able to negotiate procedural hurdles involved in a petition’s processing.

In the News

We’re happy to share two great articles from the past week that feature the work of the Lab.

The first, “The Justice Gap: America’s unfulfilled promise of ‘equal justice under law'” by Lincoln Caplan, is a longform piece in Harvard Magazine that puts the Lab’s studies of self-help materials in the context of the larger debate about how best to address the access to justice gap. It’s a great read if you’re interested in learning more about the history of legal aid and how the work of the Lab fits into that framework.

The second, “Unicorns: RCTs, the Social Sciences, and IRBs” by Tonya Ferraro, published in Public Responsibility in Medicine and Research (PRIM&R)‘s blog, Ampersand, draws on Jim Greiner’s research on the history of RCTs in the legal profession. Ensuring that studies that involve human subjects, such as the studies run by the Lab, are ethical is an important part of the legal research process that both lawyers and review boards can be unfamiliar with because of the paucity of RCTs in law. The article describes how the IRB process can adapt to meet the needs of such studies.

For those of you who don’t engage in academic research, “IRB” is short for Institutional Review Board; institutions whose faculty and students engage in research establish their own committees, which ensure that any studies involving human subjects meet federal ethical standards. (If you’re unfamiliar with IRBs and how they operate, you can learn more here.)

See any great work about access to justice in the press? Share it with us on Twitter or Facebook.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Top 10 A2J Research Priorities #9: Alternatives to Litigation

This video series describes what we consider to be the top 10 access to justice research priorities. The quick 2-3 minute videos will wake you up and get you excited about the ways that experimentation, research, and (of course) RCTs can improve access to justice.

Here’s the latest installment: Alternatives to Litigation.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Law, AI, and Justice

Yesterday, Research Director Chris Griffin spoke with three other Harvard scholars as part of a HUBweek 2017 panel sponsored by the Berkman Klein CenterProgramming the Future of AI: Ethics, Governance, and Justice. The four debated the promises and perils of using computer models and algorithms to guide legal decision-making. The Boston Globe‘s article about the event captures the core questions succinctly: “Should sophisticated computer models help judges predict which defendants are safe enough to release before trial? Or should judges rely on their own wisdom, discretion, and experience to make those decisions?”

Such questions are at the heart of the Lab’s work testing the Public Safety Assessment (“PSA”) in Dane County, Wisconsin (and potentially more sites). The PSA is an actuarial risk assessment that applies an algorithm to static criminal history factors and recommends whether and under what conditions someone should be released prior to disposition. While not an example of artificial intelligence–itself a topic for separate debate!–the PSA does raise similar questions regarding how algorithmic models should influence human decisions in law and whether or not those influences can yield more just outcomes. We hope our study will help provide some answers.

The sold-out event lasted about one hour; if you missed it live, you can catch the full conversation here!

Happy 100th, FDRP!

The Financial Distress Research Project hit a milestone this week. Our hardworking staff just randomized case number 100! And this is just the beginning. The study will continue over the next two years to randomize cases and will eventually have over one thousand study participants, generating one of the richest data sets ever used to evaluate what works and what doesn’t for people in financial distress.

While the Lab is known for its work with numbers and data, our FDRP staff also interact directly on a daily basis with individuals participating in the study. Each of those 100 is someone who will receive assistance with their debt collection case in the form of either an attorney or self-help materials. Dealing with debt in the courts is a complicated challenge involving a maze of choices. (If you’re looking for more reading on the topic, check out this recent piece in The Atlantic about the difficulties people face when trying to declare bankruptcy in Tennessee.)

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

A2J Lab featured in The Practice

The blog has been on a brief hiatus, but we’re back with some great news!

The Center on the Legal Profession (of which the Lab is proud to be a part) publishes The Practice, a bi-monthly magazine featuring topics of interest to practitioners. If you’re looking for an engaging read over the weekend, you should check out this month’s issue. That’s right—it’s a whole magazine full of stories about the Lab!

The content includes:

Note: The magazine is behind a paywall. If you register, you can read two articles for free. You can learn more about subscription options here.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

The Lab’s New Guardianship Service of Process Study Has Launched!

As of yesterday, September 5, a new A2J Lab study is in the field at the Edward W. Brooke Courthouse in downtown Boston!

CSC staff attorney Jorge Cólon couldn’t be more excited to have our hard copy self-help materials.

Together with our partners at the Boston Court Service Center (“CSC”) and the Volunteer Lawyers Project (“VLP”) of the Boston Bar Association, we are evaluating (via RCT, of course) whether self-help materials can make a difference for court users seeking guardianship over incapacitated adults or minors. CSC and VLP reported high rates of return visits from users they assisted with filling out petitions. Why? Those petitioners often got stuck trying to navigate the often-labyrinthine service of process requirements.

A previous blog entry revealed just how confusing the process of service can be. Petitioners, most of whom are not lawyers, have to:  (1) identify “interested parties,” many of whom are not obvious candidates; (2) determine the proper method of service; (3) effectuate service; and (4) return proof of service to the Probate and Family Court. Completing the process exactly as described is equally important. Unless parties complete all the steps in the correct order, the guardianship matter cannot proceed.

Cólon and his CSC colleague Carolin Hetzner are already distributing (randomly!) Blob’s latest adventures to guardianship petitioners.

Because the process is complicated and the constituencies served have limited access to legal resources (beyond the excellent help of the CSC and VLP), the Lab’s familiar promotion and development of self-help materials seemed like a natural response. The associated RCT will lead to randomized provision of printed materials (developed in large part at our first hackathon) for both adult or minor guardianship cases and in English or Spanish. Once again, Hallie Jay Pope, the intrepid leader of the Graphic Advocacy Project, designed vivid flowcharts, “happy maps,” and new manifestations of the Lab’s favorite humanoid, Blob, to enrich the paper product. In addition, minor guardianship petitioners randomized to receive the hard copy booklets will also gain access to an online tool developed by Bill Palin, the Access to Justice/Technology Fellow with Harvard Law’s clinical programs. That site walks users through their unique legal needs, much like the software pioneered by TurboTax and other online service providers. The RCT will compare rates of successful service, among other outcomes, between the treatment and control groups.

What happens now? The first steps are randomizing cases and letting users go forth with service of process; that’s the part that began yesterday. We’ll start collecting data later this month. Stay tuned here for continued updates on the project!

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.