Comparing the effectiveness of evaluating practical capabilities through hands-on on-line exercises versus conventional methods

Two interrelated methodological transformations involved in the current transition of European universities towards the European higher education Area (EHEA) are the role of applied capabilities and the evaluation process. In this context this paper presents the results of a structured comparison, throughout a five course period, of the impact of alternative evaluation methods in courses aimed at the development of applied engineering capabilities. The comparison perspective is twofold: how accurately does the evaluation method measure the competence level attained by the students, and how does it affect their active learning. The experiment was conducted in a simulation course from the industrial engineering curriculum and the aim was the evaluation of the capability of using a simulation software. Evaluation was traditionally based on a written final exam and two other evaluation methods were then introduced: Computer exam and team project assignment. The assessment of the evaluation methods was carried out by both faculty members and students (through anonymous surveys). Results suggest that both group assignments and computer exam perform far better, in this environment, than written exams. The comparison between group assignments and computer exam is less straightforward, being dependant on which criterion is being appraised.