You are here

  1. Home
  2. Impact of SoTL
  3. Impact Evaluation Framework for SoTL

Impact Evaluation Framework for SoTL

Based on a literature review and over a decade’s experience of SoTL practice in eSTEeM, we have developed an Impact Evaluation Framework (IEF) for SoTL consisting of a set of 12 impact criteria. These 12 impact criteria or facets of impact can be useful for planning for impact when designing a SoTL inquiry, for evaluating the impact of a SoTL inquiry, or for guiding the collation of evidence when evaluating the impact of SoTL activity at the institution level.

The 12 facets of the Impact Evaluation Framework (IEF) for SoTL are divided into four categories:

  1. Learning and Teaching.
  2. Transfer to others.
  3. Stakeholder benefits.
  4. Cultural and economic benefits.

The facets and attributes of the IEF for SoTL are listed under each of these four categories in the Table below.

Learning and Teaching

1. Student experience

pre-registration; induction; curriculum design; design of assessment; learning design; student engagement with course content; student engagement with the technological intervention; and student satisfaction rate.

2. Student retention and progression

student registrations; average marks as compared with previous year(s); module completion rate; module pass rate; student retention rate; and student progression.

3. Excellence in teaching

student skills-set (e.g. academic writing; critical thinking; reflection; problem-solving; group-working; digital literacy); student employability; evidence of research-informed teaching; data for assessments (e.g. UK’s TEF), programme reviews and accreditation processes; inter-disciplinary collaborations in teaching; accreditation against professional standards; informing policy development internally at the level of department, faculty or University; and informing policy development externally (in another institution or in the sector).
Transfer to others

4. Discipline-based teaching, research and practice

change in the ways in which subject concepts are taught; leading to discipline-based research; uptake of outputs in industry practice.

5. Dissemination of project’s outcomes

number of publications from the project; impact factor of individual journals or conferences; publications with students as co-authors; Google Scholar analytics or institutional analytics (e.g. OU’s ORO on downloads of reports/publications; and sharing of novel research methods/strategies for conducting SoTL.

6. Adoption of the outcomes

adoption of the outcomes internally (within the institution) or externally to improve assessment, curriculum design in the same discipline or in other disciplines.
Stakeholder benefits

7. Mutual stakeholder understanding

understanding among students, tutors, learning designers, IT support; for example, their skills, challenges, requirements; a community that SoTL creates and moving outside traditional silos.

8. Personal and professional development of project team and associated stakeholders

improved practice or personal knowledge; developing an analytical mind-set; collaborative or team-working skills; reflective skills; becoming a mentor to others; becoming a champion for SoTL; continuity in SoTL activity by individual educators.

9. Recognition of project team members and other stakeholders

career trajectory that can be attributed to SoTL such as promotions; fellowships or memberships of professional associations nationally and internationally; invited speaker to events/conferences internally and externally; public recognition through awards, publications, conference presentations; leadership roles related to teaching and membership of strategic committees; external examiner and membership of external bodies.
Cultural and economic benefits

10. Fostering of SoTL culture

stimulating interest in SoTL; inspiring others to conduct SoTL; increased involvement of students in SoTL projects; a stronger overall faculty that values teaching and student learning; renewing/raising faculty excitement about teaching and making them more aware of how they teach; a move towards staff-student collaboration in curriculum design, development and evaluation; recognition of SoTL at par with disciplinary research.

11. Financial implications

opportunities for income diversification; effect on costs of modules or programmes.

12. Funding opportunities

internal (within the institution) funding for follow-on/new projects based on SoTL project’s success; external funding (from outside the institution) for follow-on/new projects based on SoTL project’s success.

 

The impact evaluation framework for SoTL is shared under Creative Commons licence: Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0, https://creativecommons.org/licenses/by-nc-sa/4.0/)

Creative Commons License Logo