Do OU students understand the Learning Outcomes on courses in general and in T176, T192, T193, T194 in particular?

Project leader(s): Alan Yate
Theme: Supporting students
Faculty: STEM
Status: Archived
Dates: June 2019 to August 2021

Anecdotal evidence from ALs tutoring on some Level 1 Engineering modules seems to show that students on Engineering modules do not fully engage with the Learning Outcomes (LOs). This project aimed to find out if the way in which LOs are written may be a barrier to learning as LOs are a key part of module design.

Four Level 1 modules were studied (T176, T192, T193 and T194). There were four Phases to the project which ran from 2019 to 2020.

Phase I – The same six Readability Tests from four websites, giving a total of 24 tests, were used on the six LOs from module T176.These tests revealed that there were problems with the number of syllables per sentence, the length of sentences and the Reading Age needed to understand the LOs.

In addition, each module was analysed to identify whether there were assignments (TMAs and EMAs) which specifically asked students to engage with LOs in these assignments, the assumption being that students would have to engage with the LOs to complete them. We found that only T176 directly used LOs in TMAs and the EMA.

Phase II – armed with this data, we devised a student activity which was delivered face-to-face at a T176 residential school at Bath University in 2019 to collect more data. 37 students took part in groups of 4/5 and the task took about 20 minutes to deliver. Five out of the six LOs in T176 were used and a technique called a ‘Group Administered Interactive Questionnaire’ (GAIQ) (Yerushalmi, E., Henderson, C., Mamudi, W., Singh, C., Lin, S.; (2012) was used. The results were used to design our two student surveys in Phases III and IV using Qualtrics software and delivery on-line.

Phase III was a survey delivered to 376 students in 2020 as a mixture of Likert psychometric style, open ended, closed, rating type and Multiple-Choice questions. 16 questions were developed.

The Likert psychometric questions used only four choices (rather than the five usually used) to avoid respondents choosing the neutral option as a ‘sitting on the fence’ choice and forcing a decision. A delay in distribution for Phase III (COVID-19!) resulted in T176 being surveyed in mid- module, rather than upon completion. These students had also completed modules T192 – T194.

Phase IV was a second survey delivered to 1820 to students who had completed T176, T192, T193 and T194 with some minor alterations to several questions as a result of Phase III. Both surveys were then compared.

This comparison showed that many students were frustrated by the structure of some LOs in their modules, particularly in the length of sentences and the amount of information in a sentence they were expected to understand. The Phases II, III and IV therefore provided evidence on the analysis we had carried out in Phase1, particularly in the integration of LOs into assignments to improve engagement. Students also made useful suggestions on how the design of LOs could be improved. These improvements will be made available to course teams.