Transforming retention and progression in a new Level 1 course

Project leader(s): Chris Dobbyn
Theme: Innovative assessment
Faculty: STEM
Status: Archived
Dates: June 2011 to May 2014

This project was set up to evaluate the effectiveness of the models of assessment devised for the new Level 1 course, TU100: My Digital Life. Assessments and tutor guides based on novel models had been prepared by a small group of staff, including the investigators. The assessments were intended to improve retention by developing self-regulating learners capable of progressing an OU career, by offering joined-up, module-long assessment of, and feedback on, academic and learning skills – assessment for learning rather than assessment of learning.

The main objective of the project was to survey tutors’ perceptions of feedback and their rationales in providing it, in the light of this new assessment and feedback regime; a second objective was to identify foundational concepts and potential “leakage points”, (points in the course at which students seem to leave), at which support and additional material could then be targeted. The broad hypothesis was that tutor behaviour in assessment and feedback can be improved through an assessment policy specifically designed to build self-regulating learners.

Data was gathered from a series of structured interviews with a selected group of (13) TU100 tutors at appropriate points throughout the presentation; an analysis of monitoring reports and an analysis of responses to two surveys – the post-launch survey undertaken by the Module Team, and an IET (Institute of Educational Technology) tutor survey undertaken post 2012J (October 2012) and mid 2013B (February 2013), including comparison with an earlier survey of T175 tutors.

Analysis of interview transcripts revealed a clear dichotomy among the tutors interviewed, mainly along the dividing line between experienced and relatively inexperienced tutors. Tutors who had worked for the OU for longer than 5 years were revealed to be primarily concerned with whether the changes would achieve their aim of providing enhanced skills development for students and more useful assignment feedback. They were secure in their understanding of feedback and its purpose, and at ease with the four-quadrant terminology described below. The new regime generally fitted in well with these tutors’ past practices and they were generally happy with the new regime’s focus on skills. Less experienced tutors, however, seemed primarily concerned with what the changes meant for them; how the changes would impact on their workload; and whether any difficulties in implementing the new scheme would reflect badly on their performance. Among this group, conceptions of feedback were diverse and occasionally seemed incoherent. The language of marks predominated, and skills or skills development were rarely mentioned unless the interviewee was specifically asked about it.

However, all the tutors interviewed were initially enthusiastic about the greater focus on skills, and by the time they had marked 3 TMAs (Tutor Marked Assignments) were still positive about the idea of focussed skills feedback. By the end of the module most tutors were showing more confidence in their ability to give skills feedback, although some still felt that it was hard to separate skills comments out away from the question/answer content.

There was one point on which there was universal agreement: that the most problematic foundational concept and stumbling block within TU100 was Sense programming. This finding, we believe, raises questions about the way programming is taught and assessed on the module.