Abstract
This study aimed at investigating how the factors across the student and school levels would influence students’ performance on a technology-based assessment (TBA). TBAs not only enable teachers to evaluate their students’ complex abilities, but have also been adopted by large-scale and international evaluation programs in recent years. Although some factors such as students’ engagement or teachers’ beliefs about assessments have been articulated, relatively little is understood about how the factors across levels affect students’ performances on TBAs. This study thus collected data of 494 science teachers and 1774 eighth and 11th graders from 32 schools, and conducted a hierarchical linear modeling analysis to provide a more accurate estimation of the effects of the variables in each level on students’ performances. The results indicated the importance of students’ engagement in related learning activities and their computer experiences at both the individual and school levels. Additionally, although none of the teachers’ variables at the school level such as the time teachers spent on the use of various types of TBAs or their intentions to use TBAs had significant main effects on students’ performances, the teachers’ influence at the school level on students’ learning could still be found by a significant moderating effect from teachers’ usage of TBAs. Our results provide insight into how to promote students’ performance on TBAs and can contribute in various ways to future research efforts concerning the use of TBAs in classrooms.