Centre for Scholarship and Innovation
Highly Commended at the 2nd eSTEeM Scholarship Projects of the Year Awards 2019 under the category - Innovative/Original Approach to Teaching.
Students studying M250 (Object-oriented Java programming), our second year object-oriented programming module using Java, are required to complete Java programs according to detailed syntactical, structural, functional and stylistic specifications.
Although software tools exist for code syntax, functionality and style checking, tools for structural specification checking are not widely available. The long-term goal of this project is to raise awareness of these various aspects of correctness in our assessment of students’ code and to support automated assessment of these aspects of code quality for tutors and students.
The project focused particularly on the development and evaluation of a structural specification tool (known as CheckM250), deployed in the 2017J (October 2017) presentation of M250, to allow tutors to check to what extent students’ code met a specification. The tool was provided for use in the module IDE, BlueJ, alongside traditional tutor marking notes. The project also explored the use of automated marking in the module Virtual Learning Environment (VLE), for quick feedback to students, and overcame technical obstacles in this context.
Tutor surveys and interviews were used to gather feedback on CheckM250 and on other kinds of marking tool support and traditional resources.
Automated structural checks on code were found to have multiple use cases:
There was evidence of tutors favouring the use of marking tools, or of their distrusting them, or finding them an obstacle. This appeared to depend less on the tool itself than on a predisposition for or against the use of tools. Similarly, tutors’ comparative rating of tools as aids to themselves versus as aids to students appeared to depend on the tutors’ disposition towards tools.
Most tutors using CheckM250 found it to be useful, and some reported that it increased their accuracy in marking. Tutors not using the tool cited lack of time and the simplicity of the assignment it was trialled on. Some reservations were expressed about reliance on automated marking tools, both for markers and for students. The marking software was also shown to be useful in the VLE for automated student feedback.
The results provided indicators of topics that should be discussed with tutors and students in this context:
The project has also suggested ways forward in developing automated marking tools for Java code.