Are You Ready for Your Studies - Are we Assessing Students Readiness? An evaluation of the usefulness of the Level 2 ARFY quizzes

Project leader(s): Soraya Kouadri MostéfaouiOli Howson
Theme: Supporting students
Faculty: STEM
Status: Archived
Dates: November 2019 to September 2024

The aim of this project is to investigate the success of using the diagnostic AYRF ('Are You Ready For') quizzes and their role in preparing undergraduate students for their level 2 modules (TT284 (Web technologies) M269 (Algorithms, data structures & computability) and M250 (Object-oriented Java programming) with TM255 (Communication and information technologies) as a comparator non-programming module) including investigating whether any follow-up support is offered or requested by students who achieve low scores but go on to start the module.

The research data includes: two focus groups of module team members and tutors of the level two modules investigated (n=9); a survey of level two students (n=74); and a follow up in depth interview with five of the survey respondents. These three sets of data were considered separately before a triangulation of the results.

Findings show that from the students’ perspective, most survey and interview participants found that the quizzes made them less anxious and more confident to start the modules and said they would recommend using such diagnostic quizzes in other modules. However, only 62% though the quizzes assessed their readiness to study the modules and the majority of respondents felt that the content of the quizzes could be improved, making them properly reflective of module content and difficulty, and increasing the content, with the latter potentially extending the remit of the quizzes to self-testing more generally, and preparation for the modules.

From the educators’ perspective, the issue of ‘where to go next’, if a poor result is achieved and the lack of a discussion forum, where students can discuss the quizzes and their results, emerged as themes for suggested improvements. The availability of a second quiz midway through the modules’ presentation to assess the students’ progress, as well as increased module content coverage and the inclusion of softer skills such as report writing, and time management were also suggested as potential ways of better assessing the readiness of students.

A statistical analysis was carried out to measure linear correlation for each module between quiz results and final grades using Pearson’s correlation coefficient. Findings from the statistical analysis suggest it is reasonable to use the AYRF results as an indicative predictor towards final grades for three of the four modules included in this study (TT284, M269 and M250) as these all had a significant correlation at p < 0.5. However, it would not be reasonable to use the TM255 AYRF results as predictors toward the module’s final results; the far lower completion rate of the AYRF quiz for TM255 led to very little data being available for analysis and correlation at p < 0.5 was not found to be significant.