A Cautionary Tale of Tutoring Hard-to-Reach Students in Kenya
Beth Schueler, Daniel Rodriguez-Segura
What was this study about?
Covid-19 school closures have generated significant interest in tutoring to make up for lost learning time. Tutoring is backed by rigorous research, but it is unclear whether it can be delivered effectively remotely. We study the effect of teacher-student phone calls in Kenya when schools were closed. Schools (j=105) were randomly assigned for 3rd, 5th and 6th graders (n=8,319) to receive one of two versions of a 7-week weekly math intervention—5-minute accountability checks or 15-minute mini-tutoring sessions—or to the control group.
Chart notes: Treatment effects of the phone-based tutoring intervention on March 2021 in-person math exam scores by propensity to be in the March 2021 in-person assessment sample.
What did we find?
Although calls increased perceptions that teachers cared, accountability checks had no effect on math performance four months later and tutoring actually decreased achievement among students who returned to their schools after reopening. This was, in part, because the relatively low-achieving students most likely to benefit from calls were least likely to return to school and take assessments (based on our finding that the gains were largest among students who were observably similar to those who did not return). We provide suggestive evidence this was because tutoring substituted away from more productive uses of time. Neither intervention affected enrollment.
What are some the implications?
These results are not a referendum on tutoring programs as a whole. In hindsight, there were several key differences in the design of the intervention under study here and the context in which it was implemented, compared with those high-impact tutoring interventions for which researchers have documented impressive benefits. The intervention we study was light-touch relative to state-of-the-art high-dosage models and was delivered outside of school and at a distance, due to logistical and financial constraints. Nonetheless, results have important implications for leaders determining how best to spend relief funding and support students who lost learning time due to Covid or who fall behind in non-pandemic times. As interest in tutoring grows, our study highlights the need to pay careful attention to designing programs so that they are aligned with those high-impact models, to effectively target interventions and limited funds to students most likely to benefit, to weigh trade-offs between scale and intensity of an intervention, and to anticipate and minimize unintended consequences.
Full Article Citation:
Schueler, B.E. & Rodriguez-Segura, D. (2022). A Cautionary Tale of Tutoring Hard-to-Reach Students in Kenya. Journal of Research on Educational Effectiveness. DOI: https://doi.org/10.1080/19345747.2022.2131661.