Measuring the Impact of Learning Design and Course Creation (LDCC) Workshops

• Project leader(s): Tom Olney
• Theme: Other
• Faculty: STEM
• Status: Archived
• Dates: October 2019 to January 2024

Highly Commended at the 7th eSTEeM Scholarship Projects of the Year Awards 2024.

This project aimed to measure the impact that a professional development offering for academics tasked with designing open, online and distance learning called the ‘Learning Design and Course Creation (LDCC) Workshop’ had on the individuals who attended it, and the institutions they came from. To date, around 1000 participants from 9 separate OU/HEI have taken part in 37 instances of the LDCC Workshop. The participants were mostly from China, but some were also from Belarus. Of the 37 instances, 13 were facilitated face-to-face on the OU campus, and 11 were facilitated face-to-face in China. A further 9 were delivered using a hybrid approach where the participants were co-located face-to-face in China and the facilitators were online in the UK. The remaining 4 workshops were facilitated entirely online with both participants and facilitators distributed. 

The LDCC Workshop is based on the approach to learning design established at the OU by the Open University Learning Design Initiative (OULDI) and current practice in module production in the STEM faculty. It brings together open and distance learning (ODL) educational principles, tools, activities, and examples of practice currently in use at the OU which, for the purposes of this project and the deliverables it produced, are referred to as LDCC approaches. Through a series of structured, collaborative activities the LDCC Workshop adopts a social-constructivist pedagogy that challenges participants to design an ODL course of their own in a compressed timeframe which they then present to their peers. Student-focused design and quality assurance are key elements of the workshop. It has run since 2014, and has been designed and facilitated by Tom Olney, Duncan Banks, and Mark Endean who originally started this evaluation project. However, as the project has developed, other academics and researchers have become involved, notably Daphne Chang, Lin Lin, and Bart Rienties.

After delivering the first few instances of the LDCC Workshop we soon discovered that the Chinese OU institutions we were working with were looking for a way to enhance the quality of their ODL and, as in the UK, US, Australia, and Europe, recognized that learning design could provide a mechanism for this. Since the impact of professional development is contested, we wanted to create an evidence base to establish if the workshops were having any real impact and, if so, what kinds of impact they were and what was it about the workshop that was generating this. 

The evaluation was based on an iterative approach which took place during four years between 2019 and 2023 and consisted of four phases: 

• Phase 0 evaluated feedback data from 220 Chinese OU staff who had participated in LDCC Workshops prior to the start of the project.
• Phase 1 evaluated impact data from around 30 staff from six Belarusian HEIs who participated in the BELL Project in a mixed method analysis.
• Phase 2 evaluated impact data collected from 136 Chinese OU staff via an online feedback instrument.
• Phase 3 evaluated impact data from 14 in-depth qualitative interviews with Chinese OU staff.

Related Resources: 

Attachment	Size
Tom Olney, Mark Endean and Duncan Banks poster.pdf	288.59 KB

Project poster.

Attachment	Size
Tom Olney, Measuring impact of LDCC workshops. eSTEeM Final Report.pdf	164.89 KB

eSTEeM final report.

Attachment	Size
Olney_Appendix C_Deliverable 0.3_Measuring the Impact of LDCC Workshops.pdf	248.73 KB

Journal paper, Appendix C of final report.

Attachment	Size
Olney_Appendix D_Deliverable 1.1_Measuring the Impact of the LDCC Workshops.pdf	1.89 MB

Conference paper, Appendix D of final report.

Attachment	Size
Olney_Appendix E_Deliverable 1.2_Measuring the Impact of the LDCC Workshops.pdf	229.75 KB

Conference paper, Appendix E of final report.

Attachment	Size
Olney_Appendix F_Deliverable 1.3_Measuring the Impact of the LDCC Workshops.pdf	725.39 KB

Journal paper, Appendix F of final report.

Attachment	Size
Olney_Appendix G_Deliverable 2.1_Measuring Impact of LDCC Workshops.pdf	1.34 MB

Journal paper, Appendix G of final report.

Attachment	Size
Olney_Appendix H_Deliverable 3.1_Measuring Impact of LDCC Workshops.pdf	29.44 MB

Conference paper, Appendix H of final report.

Attachment	Size
Olney_Appendix I_Deliverable 3.2_Measuring the Impact of LDCC Workshops.pdf	1.43 MB

Journal paper. Appendix I of final report.

Attachment	Size
Olney_Appendix J_Deliverable G.1_Measuring the Impact of the LDCC Workshops.pdf	2.69 MB

Report, Appendix J of final report.