Video analytics: What are academics looking for?
May 26, 2021
Keywords: Teacher Dashboards, Video Analytics
Target readers: Educators; Learning Technologists; Instructional Designers
Author: Amanda White
Position: Senior Lecturer and Deputy Head (Education) – Accounting Discipline, Business School, University of Technology Sydney.
Amanda’s research focuses on the teaching and learning aspects of accounting education and academic integrity. She received the 2021 Australian Awards for University Teaching Excellence Award for Business, Law, Economics and related fields, and was runner up Australian Teacher of the Year.
Video analytics: What are academics looking for?
I’m a firm believer that video is a great way to supplement and augment my classroom teaching. But I don’t want to waste time producing resources that aren’t engaging students, or assuming that students are watching my videos end-to-end, when they bail out after 60 seconds. This makes video analytics a really important tool of the trade.
While I have always been a high user of educational video, the shift to online learning has resulted in a massive shift across higher education to video learning materials of all types – recorded lectures, short explainers or live class recordings. The question of “are they watching?” is even more important when most institutions are switching the delivery of knowledge content to online recordings (and saving synchronous or face to face time for active learning).
My “must have” features for a video analytics dashboard
The features in most LMS video analytics dashboards have come a long way in the last five years since I started digging into analytics more generally. The most developed video analytics dashboard (in my experience) is the YouTube Studio dashboard available to content creators. It is no surprise then that educational video platforms are modelling their dashboards on this. Here are the features it offers that I find most useful:
| Feature | Reason |
| Number of plays | For a quick mental comparison of the proportion of total students in the course (but it would be better to see the % students as well — except YouTube doesn’t know anything about my student cohort) |
| Number of students who have viewed | Again, to facilitate a quick mental calculation of what proportion of students in the subject/course have viewed the video (assuming the video is embedded in only one subject) — but again, a % figure would assist. |
| Average minutes viewed and % of the video viewed per unique student | To help me design better future videos |
| The number of views on a date timeline | To visually identify when students are watching the content – just prior to class? After class? |
| Filter function | Video analytics are stored per video, and since videos are likely to be used for multiple semesters, setting the date range is critical to see just the semester being taught. |
Features such as “impressions” (which is a fancy way for saying when students see the video embedded on a page) are less useful for educational institutions. Every time the student goes to a LMS page and scrolls past the video to get to another element (text, a different video, H5P interactive etc) – it will register an “impression”.
The Kaltura analytics are available on a per-video basis – there is no overview of analytics like you’ll see on YouTube. An example is below.
Kaltura overview analytics for a video
What is missing and where video analytics need to go from here
Analytics per video are great, but imagine you have a 12-week subject (our term for a course at UTS) that has 5-8 videos per week – that’s 60-96 videos that you’ll need to individually click on to see what is happening!
Video analytics now need to evolve to include aggregated dashboards and linkages to base LMS analytics data. LMS base data includes reporting on overall levels of student activity including pages viewed, hours spent on the platform and assignments completed. Integrating video data into these dashboards would be extremely beneficial. This would allow educators to review student video usage at a glance. My ideal aggregated dashboard would be something like the image below.
Amanda’s ideal overview dashboard
For a specified time period (defaulting to the most recent 7 days) – you could see total activity and average activity for students. Filters are of critical importance – the first being the date range, but the other being the subject where the video is embedded. Where videos are in multiple subjects, this would allow you to parse out the video analytic data for your students only.
The bottom left box displaying what students are watching now would help academics understand student behaviour. I only have 3 videos in that list – but I would envisage this as a Top 10 list. This type of display would also be very useful during the period where students have a study break before major assessments or exams – pin-pointing what topics or components students are revising the most, and potentially giving you insights into what you might cover in exam Q&A sessions or drop-ins.
The bottom middle box displays students who have not engaged with any of the video learning content. As much as we want to know what students ARE watching, we also want to know which students are not watching at all – a form of exception reporting – that would help us reach out to students who may be at risk of falling behind.
The bottom right box is useful where in-line video commenting has been enabled. I love this feature where I have seen it enabled, because it allows the video to become more interactive with questions and further commentary from students and staff. But knowing that these comments are appearing is critical – otherwise they end up as questions with no responses, resulting in student frustration.
So this is my ultimate video analytics dashboard – is there anything I’m missing? What would you add or delete? What do you think our chances are that we’ll have these types of tools from educational video hosting vendors?
I’d love to hear from you!