The concept of the NEWTON project is based in an identification of the challenges and barriers in creating engagement and enthusiasm amongst STEM students and recognises the need to stimulate and embrace the type of change that innovative technologies can bring. Thus, NEWTON focuses on employing novel technologies to increase the quality of the learner experience, improve learning processes and increase learning outcomes. One of the major challenges of the project is to validate the platform impact and the effectiveness of the teaching scenarios in terms of learner satisfaction, improvement of the learning and teaching experience, and the evaluation of underlying technology through European-wide real-life pilots. In this paper, we describe briefly the criteria to be taken into account for technological learning systems evaluation from a pedagogical view point. After setting out the different dimensions of learning to be evaluated, we describe the iterative approach adopted to ensure that there is a continuous feedback loop created thereby ensuring that the results of all pilot activities are shared and are informing future iterations of the NEWTON solution. Finally, we provide a few details of the work plan designed to guide all stakeholders involved in the project pilots. This work is dedicated to teachers and teaching organizations willing to engage learners in STEM education.