Using a Factorial Design to Maximize the Effectiveness of a Parental Text Messaging Intervention

Catherine Armstrong Asher, Ethan Scherer, James S. Kim

PDF Version

What features of text messaging campaigns for early elementary families might increase their effectiveness?

Text messaging interventions are an increasingly popular way to support students and their families. We compared how three features of text messages, sent to parents, affect the reading behavior and test scores of their early elementary school children:

  • Personalization: Messages containing personalized and up-to-date information about students’ summer reading vs generic reading and app information
  • View of Reading: Messages emphasizing a specific view of reading: reading as entertainment, reading as building skills (i.e., instrumental reading) or a combination of both views
  • Goal setting: Messages with a way to set and monitor progress toward a reading goal vs not having this option 

We randomly assigned 4,993 families of elementary students to receive a combination of these three message features over nine weeks of summer vacation. This allowed us to see which features worked on their own, and which worked in combination.

Personalized messages increased use of a reading educational app the most, but all message types increased it by some amount.  

The graph on the right shows the percent of students who ever logged into a free educational reading app provided to study participants, and the x-axis shows the view of reading emphasized in the messages those students received.

Compared to generic messages, personalized information messages increased usage of a reading app. All personalized messages (in green) increased login rates relative to generic messages with no personalization (in blue). We found similar results for time on the app and app activities completed.

We did not find evidence that the view of reading or goal-setting alone impacted reading amounts on the app.

More broadly, students of parents who received any type of text messages were much more likely to use the reading app than families who received no messages at all (purple dashed line).

Only some messages increased student reading performance.

We also measured the effects of text messages on student test scores in the subsequent fall. Personalized messages were more effective than both generic messages and receiving no messages. However, we found the generic messages, in most cases, performed worse than sending no text message at all. We also found the view of reading emphasized in the messages could enhance or detract from the effects of personalized information. Receiving a mix of skill-focused and entertainment-focused messages magnified the positive effects of personalized information when compared to personalized messages that focused on skill building alone.

Family text messaging campaigns have the potential to improve student literacy outcomes, but the specific features and content of the messages are necessary for determining both the direction and magnitude of those effects.


Full Article Citation:
Asher, C. A., Scherer, E., & Kim, J. S. (2022). Using a Factorial Design to Maximize the Effectiveness of a Parental Text Messaging Intervention. Journal of Research on Educational Effectiveness, 1-26. DOI: https://doi.org/10.1080/19345747.2021.2009073.

Share this post:

Comments on "Using a Factorial Design to Maximize the Effectiveness of a Parental Text Messaging Intervention"

Comments 0-5 of 0

Please login to comment