Investigating Efficacy, Moderators, and Mediators for an Online Mathematics Homework Intervention
Summary by: Muqi Guo
ASSISTments is an online math homework intervention developed to target formative assessment. Teachers and students can assign, do, and review homework using the ASSISTments platform. While doing homework in ASSISTments students receive immediate feedback on their work, opportunities to improve their answers, and hints and tutorials for select problems. Teachers receive a report that identifies which assigned homework problems were difficult for students, along with common wrong answers.
What are the impacts of the ASSISTments intervention?
This study found, for a collection of 43 public schools of Maine, that ASSISTments improved students’ math performance. Students in schools assigned to use ASSISTments (the intervention group) had higher average end-of-the-year math test scores than students in schools not assigned to use ASSISTments (the control group). The ASSISTments intervention effect was larger compared to the average effect of formative assessment interventions and of computer-based interventions in previously published studies. Furthermore, ASSISTments had a gap-closing effect: students with lower prior math achievement benefited more from the intervention than students with higher prior math scores. Additionally, the intervention led teachers to target instruction to students’ needs, although there was no further evidence that targeted review practice is what caused students’ improved math performance. The findings likely apply to most classrooms and schools in the State of Maine.
How did the authors do this study?
This study was a school-level, randomized controlled trial. Schools were randomly assigned to immediate use of the ASSISTments intervention (intervention group) or delayed use of the intervention (control group: “business as usual condition”). Data collection occurred over a full school year, with some schools participating in 2013-14 and others participating in 2014-15. A total of 43 schools, 87 teachers, and 2,769 7th-grade students provided data to the study.
The main outcome of interest was students’ 7th-grade math achievement. To investigate ASSISTments’ impact on math achievement, the authors compared the average math test scores of students offered ASSISTments to those who were not offered ASSISTments, controlling for individual-, classroom-, and school-level characteristics. The authors also compared the magnitude of the intervention’s impacts across subgroups of students defined by policy-relevant variables, including prior math achievement level (based on their 6th grade scores), exposure to poverty (free and reduced-price lunch status), or special education needs (individualized education program status). The authors further analyzed whether an improvement in the homework review practice of teachers in the intervention group was responsible for (or mediated) the effect of the intervention on students’ math scores. Finally, to assess the generalizability of impact findings to schools in Maine and schools in other states in the U.S., authors computed a "generalizability index" which assesses how similar the schools in the sample are to other schools based on schools’ characteristics (e.g., school size, characteristics of the school population, urbanicity, district size, and percent unemployment).
Full Article Citation:
Robert Murphy, Jeremy Roschelle, Mingyu Feng & Craig A. Mason (2020). Investigating Efficacy, Moderators and Mediators for an Online Mathematic Homework Intervention, Journal of Research on Educational Effectiveness, 13:2, 235-270, DOI: 10.1080/19345747.2019.1710885