Making Learning Analytics “Irresistible”: Part 1

For over a decade, learning analytics captured the imagination of instructors, students, educational administrators, and vendors, promising to help better understand and optimise learning. However, there are limited stories of large-scale success.

As learning analytics practitioners and researchers, we must use data to positively impact learners. Practically, we need to demonstrate return on investment to our stakeholders. Politically, governments are calling for more translational research. And frankly, we need to ensure that learning analytics stays relevant and meets the hope and hype of those at the centre of our work.

At the end of the day, learning analytics will be seen by instructors as another tool in their technological toolkit, and by students as another step in their learning journey. We'd love to have instructors tell us,

“the system is so well developed; the navigation is so transparent and self-explanatory and, most importantly, the functionality is so irresistible, and the applicability is so high that the temptation was too strong.”

Far from fiction, these words were expressed by a unit coordinator from the University of Sydney Business School about the learning analytics platform, SRES, in place across the university. So, how can we design learning analytics and its implementation so that instructors can't wait to use it and tell their colleagues about it?

What makes innovations irresistible?

When I first discovered the field of learning analytics around 8 years ago, Leah Macfadyen and Shane Dawson's seminal 2012 paper Numbers are not enough introduced me to Everett Rogers' Diffusion of Innovation research. Rogers suggests that individuals move through five stages of an 'innovation-decision' process, where they first gain knowledge of the whys, hows, and whats of an innovation and then decide whether to adopt it into their practice.

The characteristics of an innovation play a crucial role in the innovation-decision process. Importantly and perhaps obviously, the innovation needs to allow a tangible improvement over an individual's current practices and meet their felt needs. Termed 'relative advantage', for instructors this might be to reduce workload while enhancing student support and feedback, or to give their teaching team the right data to foster positive relationships with students, or to electronically capture data that was physically recorded or otherwise siloed or unavailable.

Likewise, the innovation also needs to fit with current practices and values so that it is more understandable and applicable. That is, the innovation needs to have high 'compatibility' with instructors' beliefs and pedagogical approaches such as supporting students, promoting self-reflection, monitoring engagement, or fostering positive teacher-student connections. Compatibility can also be with more technical elements such as using spreadsheets or an LMS as part of teaching.

I’d like to posit that the innovation-decision process is central to making learning analytics irresistible to the people who are most deeply involved in designing and delivering education: the instructors themselves.

Case study: two LA systems that address instructors’ and students’ needs

AcaWriter (University of Technology Sydney) and the Student Relationship Engagement System (SRES) (University of Sydney and University of Melbourne) are two learning analytics software packages that have received strong uptake by instructors. They do very different things, but neither are 'traditional' learning analytics tools in the sense that they don't present dashboards of ‘data exhaust’ or demographics. Instead, they start from the pedagogical needs of instructors and students. Here I'd like to briefly use key components of the 'irresistibility model' (for lack of a better phrase), built upon Roger’s diffusion of innovation model, to analyse this success.

Learning Analytics Platforms:

For the rest of this short piece, I’ll focus on the case of SRES (but see the end for more on the adoption dynamics of AcaWriter). The ability for instructors to only use certain functionality to address immediate needs meant that they could perform a 'limited trial' (per Rogers) and then ramp up the complexity of their use once they became more familiar with the software. For example, many instructors start using SRES just for attendance/participation tracking and simple email personalisation, but after a semester or two they start building marking forms and feedback portals to be embedded into LMS using LTI. From simple origins in 2012 as a mail merge and attendance capture tool, SRES has grown at the University of Sydney to cover almost all of our 70,000+ students, is used by 2,000+ staff, has 55,000+ active monthly users (students and instructors), and is being used at four other Australian institutions under a free licence.

“I think it's the first time that an online system, in particular a University one, aligns so exactly with what I was looking for, and has all the functions I was looking for out of the box.” Unit coordinator, Engineering.

Importantly, part of SRES' growth has been because the software itself has supported diffusibility, and an ecosystem has grown around it to support diffusion. Re-invention has been key to enabling this; Rogers writes that innovations that are more complex, have many applications, and can solve a wide range of problems are more likely to be re-invented. Moreover, a higher degree of re-invention leads to faster and more sustainable adoption of innovations. Re-invention has seen instructors turn SRES into an electronic photo book for teaching assistants to get to know their class better. It has seen SRES be used across a whole faculty to run objective structured clinical examinations and expedite personalised feedback from these. More recently (and strangely), re-invention has also seen SRES being used for university-wide contact tracing during COVID-19 because of its QR code functionality.

Re-invention, a key part of the innovation-decision process, has led to both new software features and new pedagogical approaches springing from (and be shared amongst) SRES’ expanding user base. Far from its simple beginnings as a mail merge and attendance tool, SRES now helps instructors get to know their students better, supports reflection, self-regulation, and teacher-student relationships, scales personalised feedback, assists with cohort management and support, and a whole lot more. The chart below outlines the co-evolution of SRES alongside instructors' needs and demands of data collection, curation, analysis, and use.