Impact Evaluation Initiative

The impact evaluation initiative has involved conducting impact evaluation and analyses of 16 Scholarship of Teaching and Learning (SoTL) projects in Faculty of STEM of The Open University (OU), 15 of which were funded and supported by eSTEeM, the OU Centre for STEM pedagogy, and one was supported by the School of Mathematics and Statistics.

The impact of a SoTL inquiry implies demonstrable benefits to learning and teaching that are directly attributable to that project. Impact of SoTL answers the question: what has changed because of this SoTL inquiry? What new insights will other educators gain from this SoTL inquiry? How can the institution put the outcomes of the inquiry into use? What are the current or emerging debates in higher education that this inquiry contributes to?

The impact evaluation has involved applying an Impact Evaluation Framework (IEF) to each of these 16 projects to assess their impact. The 16 projects have been documented as case studies which focus on the impact of the individual SoTL projects, that is, what has changed because of each of these SoTL projects.

Team

Shailey Minocha and Trevor Collins, Faculty of STEM; and Directorate, eSTEeM, the Centre for STEM pedagogy, The Open University, UK.

If you have any queries or comments on this project, please contact the eSTEeM’s directorate at this email address: esteem@open.ac.uk

Aims

The aims of the impact evaluation initiative of SoTL projects were to:

• capture the impact of a selected set of SoTL projects using the IEF,
• identify the facilitators and barriers of generating impact in SoTL projects, and
• derive strategies for supporting SoTL and for planning and evaluating the impact of SoTL.

Methodology

A case study methodology has been employed to generate an in-depth understanding of the impact of each of the 16 SoTL projects and to present the impact of these SoTL projects as case studies.

Although each of the SoTL projects had several resources (for example, end-of-project report, publications, posters) from which the case study could have been written, the impact narrative was developed in collaboration with the project team/leads. The IEF was applied to guide the impact evaluations and the project team/lead validated each case study.

The seven pedagogical themes underpinning the 16 SoTL projects and case studies are: accessibility, assessment, authentic experimental and practical learning, building digital skills, projects that have made an impact on the way a discipline is taught, supporting student journey, and student voice.

Outputs

The impact evaluation initiative has the following deliverables:

Executive summary

The ‘Executive Summary’ provides an overview of the initiative, and the key outputs of this project: the ‘compendium’ of case studies’ and the ‘guide for educators’, and two workbooks as supplementary resources to the guide.

Publication: Executive summary

Minocha, Shailey and Collins, Trevor (2023) Impact of Scholarship of Teaching and Learning: An Executive Summary. The Open University, Milton Keynes, UK.

Available at: https://doi.org/10.21954/ou.ro.000155bf

A compendium of case studies

The ‘compendium of case studies’ includes the 16 case studies. The case studies and reflections on SoTL practice in each of these SoTL projects demonstrate the role of SoTL and its impact on educator’s practice and on student learning.

Each impact case study discusses the context and aim of the SoTL inquiry, the underpinning research, the findings, details of the impact, and reflections on SoTL practice (guided by the principles of SoTL). Each case study ends with a quote from the project team, which captures their reflections on the project and its impact. The project resources including end-of-project reports, videos, papers, presentations and posters, and references to the literature are listed at the end of each case study.

Publication: A compendium of case studies

Minocha, Shailey and Collins, Trevor (2023). Impact of Scholarship of Teaching and Learning: A compendium of case studies. The Open University, Milton Keynes, UK.

Available at: https://doi.org/10.21954/ou.ro.000155c0

A guide for educators and two workbooks

The impact evaluation of the 16 SoTL projects and subsequent analyses and literature review has resulted in several outputs that are captured in the ‘guide for educators’. The guide includes:

• the identification of key enablers and strategies for the generation of impact;
• an understanding of eSTEeM’s role in supporting SoTL in STEM at the OU;
• a toolkit for planning, generating, monitoring, and communicating impact from SoTL; and
• a list of challenges to generating impact of SoTL.

There are two workbooks that accompany the guide:

The ‘Planning for SoTL impact workbook’ has been designed around the ‘Theory of Change’ to guide the educator to systematically plan a SoTL project for possible impact.

The ‘SoTL impact evaluation workbook’ is designed around the Impact Evaluation Framework (IEF) which we have employed in this guide to conduct impact evaluation of SoTL projects. This workbook can be used by educators for planning, reflecting on, monitoring, and evaluating impact of a SoTL inquiry, and recording the evidence of the impact.

Publication: A guide for educators

Minocha, Shailey and Collins, Trevor (2023). Impact of Scholarship of Teaching and Learning: A guide for educators. The Open University, Milton Keynes, UK.

Available at: https://doi.org/10.21954/ou.ro.000155c1

The workbooks are available as supplementary resources to the guide at the above link.