With the recent introduction of computer science in elementary school, teachers must monitor their students’ progress in a subject known for both its creative and challenging nature. Assessing a vast number of diverse solutions is a time-taking challenge and, without the help of automation, some teachers may be tempted to resort to traditional assessment techniques that are easier to verify but do not provide the same possibilities for constructionist learning. We present a tool to create and conduct custom programming assessments in turtle graphics. Solutions are verified by pixel-wise comparison of student and sample solutions while also considering constraints on the set of possible solutions. The tool has been deployed in the XLogoOnline programming environment and we are planning to further analyze student performance in practice.