Why RCTs? Part 6

How helpful is that Head Start?

lady_bird_johnson_visiting_a_classroom_for_project_head_start_1966

Lady Bird Johnson Visiting a Classroom for Project Head Start, 1966

Education reform is tricky no matter the level or issue targeted. But one area of reform subject to fairly constant momentum is Head Start, especially under the Obama Administration. This program, established in 1965, addresses some of the early developmental and school readiness gaps for low-income children, from birth to age five. The million-dollar question (for a many million-dollar program) is: does it work?

Before answering that question, we have to define our terms. What do we mean by “work”?  There are numerous measures of success with which one might be concerned: breaking the cycle of poverty (as was part of Head Start’s original mission), improving mental health, enhancing happiness among parents and children, or educational outcomes five or ten years down the road. A 2016 NPR article details the measurements that Head Start itself has prioritized. Recently enacted government standards now focus on kids’ progress in:

  • physical development
  • the health of their attachments to and interactions with adults and peers
  • their ability to manage their emotions and attention, both to persist and to be flexible;
  • language and literacy; and
  • basic cognition, which includes early mathematics and science reasoning.

The head of a Head Start funding organization boasted in the NPR piece: “[W]e’re very proud that we’re kind of data crazy at Children’s Aid.” The U.S. Department of Health and Human Services (HHS) funds “extensive research” into the effectiveness of Head Start. In an attempt to better evaluate Head Start outcomes when programs can vary considerably across the country, new standards for evaluation were implemented in 2016, after the first major revision to Head Start since 1975. The push to collect data on Head Start has not been a necessarily welcomed one: some are concerned that it makes the program resemble high-stakes testing.

A study of Tulsa’s largest Head Start program found “clear benefits,” for children attending Head Start, including higher test scores through eighth grade and lower rates of chronic absenteeism. In addition, benefits extended into middle school, when most fade-out is expected to occur. The study found, however, that the effects of Head Start were not the same for all children. Latino students who did not speak or read English improved considerably, while African-American boys did not appear to gain much from participating in Head Start.  Importantly, the study was not an RCT, and some argued that the Tulsa Head Start program was unusually primed for success. It enjoyed more money and talent that most programs don’t necessarily have. In another non-RCT study, well-known Princeton economist Janet Currie and co-authors studied the long-term effects of Head Start. They found that white participants in the program achieved more schooling and higher earnings in their twenties. Black participants were less likely to be involved in crime. Study participants were not selected at random, however, which makes the results somewhat less conclusive despite controlling for many confounding variables.

The Impact Study, conducted by HHS itself, found positive Head Start effects. Any improvement, though, seems to diminish and eventually disappear after first grade. In fact, this is one of the biggest criticisms of Head Start- observed benefits fade far too quickly. Some point to deteriorating school conditions as the root cause for impact fade-outs rather than the program itself. The Impact Study was a longitudinal, randomized evaluation that followed over 5,000 preschool children for several years after their participation in the program. Some have been critiqued its features, including partial contamination of the control group, and measuring “immeasurable” variables for determining success such as “grit,” socio-emotional development, or parenting.

These numerous conflicting conclusions from studies on Head Start, which should be the “ideal laboratory” for studying educational reform, highlight the need for more rigorous evaluation. I have documented in this series how RCTs can improve our understanding of how well new interventions work. A key to their success is successfully connecting experimental design to the outcomes we care about. So, in addition to changing the methodology for Head Start studies, the existing lot suggests another homework assignment for education researchers: developing more reliable and concrete dependent variables.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Leave Comment

Your email address will not be published. Required fields are marked *