Over the past decade, students’ use of computers has increased sharply, particularly for writing and research (Becker, 1999; Russell, O’Brien, Bebell, & O’Dwyer, 2003). At the same time, the use of large-scale tests to make decisions about schools and students has exploded. But, in addition to making decisions about students and their schools, results from large-scale tests are also used to assess the impact of computer-based technology on student learning. As one example, Wenglinsky (1998) used test scores from the National Assessment of Educational Progress (NAEP) mathematics test to examine the relationship between computer use during mathematics instruction and mathematics achievement.
Clearly, the use of standardized test scores to assess the effects of technology is attractive for at least two reasons. First, the public and political leaders tend to accept standardize tests as valid and reliable measures of student achievement. Second, these tests provide a common measure across large numbers of students. Yet, despite these advantages, using test scores from standardized tests to assess the impact of technology on student learning can be problematic.