Come to Test & Learn from the new Equity, Diversity and Inclusion Learning Analytics Dashboard

Miriam Fernandez, Paul Mulholland, Vaclav Bayer, Martin Hlosta and Christothea Herodotou, The Open University, UK  

Email: miriam.fernandez@open.ac.uk

Session recording

Presentation

Click to download the presentation Come to Test & Learn from the new Equity, Diversity and Inclusion Learning Analytics Dashboard (.pptx)

Abstract

The current degree-awarding gap for a ‘good module pass’ at The Open University (OU) is 19.3% for BAME students and 31.1% for Black students compared to White students. The factors behind these awarding gaps are, however, not fully understood. Research conducted in 2019 over The Open University modules showed that this is not a matter of effort spent, since BAME students spent 4-12% more study time to achieve the same academic performance as White students.   

In this workshop, we present our developed Equality Diversity and Inclusion (EDI) Learning Analytics (LA) Dashboard. This dashboard was developed following a previous analysis of existing dashboards currently used at the Open University and an interview-led requirements-gathering process from 13 different university stakeholders.  

We will first present our conducted analysis of the five different analysed dashboards  (APS Dashboard; Qualification & Progression student profile dashboard; Module Performance Self Service; ASSIST; and Curriculum Analytics Tool) as well as the results of our requirements gathering from the selected stakeholders.  

We will then present the designed and developed EDI dashboard and showcase how it can support the automatic identification of modules and materials where problems emerge. We will describe how the dashboard can help university stakeholders, particularly module chairs, by highlighting the specific parts of the module where awarding gaps are increasing and by providing key evidence from existing scholarship investigations conducted by the OU that may help to understand and address those gaps.  

A hands-on session will follow up on this presentation. We will allow time for participants to conduct an awarding gap investigation of different OU modules using the EDI dashboard followed by a focus group discussion. During the focus group, we will ask participants to describe their findings about the various modules they engaged with and to reflect on how such a dashboard could be integrated into existing stakeholder practices.  

At the beginning of the workshop, participants will also be asked to complete a brief online questionnaire. With this form, we aim to collect information about: (i) their role (AL, Module Chair, etc.), (ii) the tools that they currently use (if any) to investigate existing awarding gaps, (iii) the problems they encounter when using such tools (if any), (iv) their perceptions of the EDI dashboard, and (v) possible developments (requirements) that they may want to see in future versions of the dashboard.  

An analysis of both the questionnaire and the focus group will then be feedback to participants (if they wish to be contacted by the research team). 

Data Protection and ethical approval will be requested before this workshop in collaboration with the Research Ethics team at The Open University to ensure that both, the questionnaire and the focus group are carefully designed and that participants are aware of the purpose of the study, how the collected data will be collected, stored and analysed, and how they can give or withdraw consent.  

References

• Open.Ac.Uk, 2023, Access and Participation Plan 2020-2025.pdf
• Nguyen, Q., Rienties, B., Richardson, J.T.: Learning analytics to uncover inequality in behavioural engagement and academic attainment in a distance learning setting. Assessment & Evaluation in Higher Education 45(4), 594–606 (2020)

Vaclav Bayer

Projects technical lead, OU Analyse (KMi): The Open University

Vaclav has worked at the Knowledge Media institute on different successful projects for the last six years. He was a Web developer on CORE - the world's largest collection of open-access research papers. He is currently part of the OUAnalyse team as a technical lead of the project. As a part-time PhD student, he investigates how Learning Analytics can be used to target awarding gaps, which is a severe problem occurring in Higher Education institutions across the whole UK. He is currently focusing on developing an Equity, Diversity and Inclusion dashboard to spotlight awarding gaps across diverse groups of students within their modules and help understand the driving factors of awarding gaps.

Prof Miriam Fernandez

Professor of Responsible Artificial Intelligence: The Open University

Prof Miriam Fernandez is Professor of Responsible Artificial Intelligence at the Open University.  She applies research and technology development to tackle the ethical and societal challenges posed by Artificial Intelligence (AI). Her research focuses on addressing biases, inequalities, and online harm. She has authored more than 100 scientific articles, including publications in the highest quality conferences and journals in her area of research, and acquired numerous external grants. Her commitment to education is demonstrated by the leadership of OUAnalyse, a strategic Open University project to apply machine-learning methods for the early identification of students at risk.

She is Equality and Diversity Champion at the Open University, editor of several top journals in the domain, Work Package/Project leader in large international and national projects, and has filled multiple prominent roles in organising some of the most influential conferences in her scientific field. 

Paul Mulholland

Senior research Fellow (KMi): The Open University

Paul Mulholland is a Senior Research Fellow in the Knowledge Media Institute, The Open University. His research interests include Human Computer Interaction, Technology Enhanced Learning and Digital Humanities. As part of the European Union Horizon 2020 SPICE project he is working on tools and methods to support Citizen Curation in museums. Within the European Union Horizon 2020 Polifonia project he is working on creating novel ways of interacting with musical heritage. He has published over 100 conference and journal articles. He is Deputy Editor of the International Journal of Human Computer Studies.

Prof Christothea Herodotou

Professor of Learning Technologies and Social Justice, The Open University

Christothea Herodotou is a Professor of Learning Technologies and Social Justice at the Open University UK. She has expertise in the design and evaluation of learning technologies that promote social justice, including learning analytics dashboards, citizen science websites, mobile game applications. She is the academic lead of nQuire, developed in partnership with the BBC, which supports the engagement of the public with scientific practices. She is the academic co-lead of the Early Alert Indicators project that uses predictive learning analytics to identify and support students struggling with their studies.

Her most recent grant by Horizon Europe aims to devise sustainable models of education based on design thinking and digital technologies. She has authored more than 100+ publications mainly as a first author. She is a Senior Fellow of the Higher Education Academy (SFHEA) and an elected member of the International Society for Artificial Intelligence in Education (IAIED).

Martin Hlosta

Senior Researcher: Swiss Distance University of Applied Sciences (FFHS), Switzerland and Visiting Researcher (KMi), The Open University

Martin Hlosta  is a Senior Researcher at the Institute for Distance Learning and eLearning Research (IFeL) at Swiss distance university of applied sciences (FFHS) and a visiting researcher at KMi, OU. Before joining FFHS, he led OUAnalyse at OU - a Predictive Learning Analytics project deployed in all undergraduate courses, improving student retention, teachers practice. Currently, he is leading research and teaching in Learning Analytics at FFHS. His most recent project funded by Unity and Meta to target inequalities in education explores how VR and enhanced analytics for reflection can help teachers in South Africa.