This data-driven paper quantitatively evaluates software testing behaviors that students exhibited in introductory computer science courses. The evaluation includes data collected over five years (10 semesters) from 49,980 programming assignment submissions by 883 different students. To examine the effectiveness of software testing behaviors, we investigate the quality of their testing at different stages of their development. We partition testing behaviors into four groups according to when in their development they first achieve substantial (at least 85%) test coverage.

The study reveals significant results regarding effective and ineffective testing behaviors. A within-subjects comparison finds that higher coverage in early development is associated with higher quality code and with completing work earlier. Post-hoc analysis also suggests that the relationship between early testing and positive outcomes is independent of time management and effects of individuals' abilities. However, roughly 76% of students exhibit different testing behaviors on different assignments, demonstrating an opportunity to foster better, more consistent testing habits among computer science students.