Abstract
This study investigated a rule-based automated prompting system for detecting and guiding at-risk learners on an online mathematics practice platform in South Africa. In this prompting system, textual prompts were embedded in the platform to encourage learners to engage with revision resources when they appeared to be struggling. First, a simple rule-based approach was used to detect struggling learners, and its accuracy was determined for different timings. Then, a randomised control trial with 3 751 Grade 8 learners from 730 schools across all nine South African provinces was used to investigate how the timing of the prompt affects its efficacy. The impact of other contextual factors was also explored. These included school wealth, if learners had previously accessed revision resources, if they were new to the platform, and the number of prompts they had already seen. Notable results are that more aggressive (faster) prompt timings result in significantly more learners engaging in revision compared to more conservative (slower) timings but with higher false positive rates; learners from poorer schools and learners who have already engaged in revision activities on the platform are more likely to respond positively to revision prompts; and learners habituate to prompts with an exponential drop in the response rate for every additional prompt they see. This study has implications for the designers of online practice platforms as well as for future research in this area, where few studies pair the detection of at-risk learners with a specific intervention.