A recent emphasis has been placed on embedding automated assessment systems within open-ended virtual 3D environments. Several supporting assessment models have been identified to guide the design of technology in support of both teachers and learners, most notably within game-based learning. Current research in automated assessment systems indicates that completeness, accuracy of performance, and timeliness are useful indicators of student learning within simulations. However, a better understanding is needed to guide designers and developers of these systems when facing decisions about where and how to effectively implement the technological structures that support key functions. This chapter identifies a set of design decisions as the result of building an automated assessment prototype within an open-ended 3D environment. The results are expanded through an implementation in a context beyond the prototype, further informing the refinement for an effective development strategy and testing the validity of the proposed system.