Learning with and from Artificial Intelligence-Driven Analytics

Artificial intelligence (AI)

The phrase “Artificial Intelligence” (AI) is often associated with ideas of humanoid robots, self-driving cars, or virtual assistants like Alexa and Siri. Rarely do we think of classrooms as primary examples of AI, but interdisciplinary innovations across fields such as Computer Science and the Learning Sciences are using AI to reshape how we measure learning in educational settings. So what does this mean for the future of education? How can we design AI that is safe, equitable, and effective in both assessing and promoting learning? The answers to these questions can be found in research at the intersection of AI and learning analytics.

What is AI?

AI can take on many forms but generally involves machines that are able to make decisions and adapt in similar ways to people. People interact with their environments based on different experiences over time and “learning” from the outcomes of those experiences (e.g., when we drive to work we first try to take a direct route and eventually discover that some roads have significant traffic during rush hour). We then use this information that we have learned to inform our future decisions (e.g., we may decide to take a longer, less direct route in order to avoid traffic), but sometimes these decisions are far from perfect depending on the context and unexpected obstacles (e.g., there happens to be construction the day that we take the less direct route and so we are still late to work).

Just like humans, machines can be trained to make decisions by providing them with experiences (i.e., in the form of data) and the corresponding outcomes associated with the experiences to create a model (i.e., representing relationships between certain features of the data and outcomes). The machine then applies the model to make decisions and predictions about outcomes of new experiences based on the features of the new data. These AI models can be applied to help accomplish complicated tasks across contexts, including in classrooms. Specifically, AI is often used to make decisions and predictions about student learning outcomes in educational settings by informing learning analytics.

AI-Driven Learning Analytics

One reason that AI is particularly useful in educational settings is because it can drive the assessment of complex student competencies that would otherwise be challenging and time-consuming  to measure. For example, it takes hours to manually grade middle school students’ argumentative essays, particularly because there is not just one “correct” answer or way of organizing the essays. Researchers, however, have created AI-driven models that can automatically score students’ essays for complicated areas such as students’ use of evidence to support claims in their writing as well as the organization of the ideas within the essay.

AI analytics can also be used to inform individualized support to students and displays of student performance to teachers. For instance, if multiple middle school students are completing a scientific lab experiment simultaneously, it can be difficult to monitor each students’ progress and provide personalized support to students at the appropriate time. AI analytics within virtual lab environments provide an opportunity to automatically score student progress and correspondingly provide personalized support to students in real-time through an AI tutoring agent. The AI analytics can also be used to keep teachers informed of student progress through a dashboard that provides real-time alerts on student difficulties as well as other essential visualizations and data on student progress.

These are just a few examples of the cutting-edge innovations emerging at the intersection of AI and Learning Analytics. It has been exciting to see the potential of these tools to support instruction and learning in classroom settings, but, just like people driving to work, AI technologies are not perfect. Let’s think through some of the challenges that come with implementing AI innovations in educational settings.

Challenges with AI in Education

It is critical to keep in mind that all initial stages of AI technologies are created by humans, which means that these tools are susceptible to human biases and error. Correspondingly, some of the greatest challenges with creating AI for educational contexts are that people must decide: the sample of which people’s experiences or data are used to train the AI model, the types of data to include in the model, the approach that is used to model the data, and how to define outcomes (i.e., decide what is a “correct” versus “incorrect” student answer). All of these decisions (and more) impact the way that AI analytics evaluate student learning, which can have a substantial impact on critical educational outcomes for students. Many AI researchers have emphasized the importance of keeping humans involved in final decision making to help mediate some of these issues.

Another core challenge with AI analytics in classrooms is privacy concerns. While using different types of data (e.g., voice data, eye movements, facial expressions, etc.) can help to make AI models more powerful in certain cases, these types of data are also extremely personal. Therefore trust is needed in terms of how data is collected, used, and stored. Establishing standards for data use and implementation of AI technologies requires cross-disciplinary efforts.

Interdisciplinary Collaboration is Essential

There have been efforts across fields and institutions to promote interdisciplinary collaboration that is addressing the challenges faced around AI in education. The National Science Foundation has recently funded several institutes including the NSF AI Institute for Student-AI Teaming (iSAT), NSF AI Institute for Engaged Learning (AIEngage.org), and AI Institute for Adult Learning & Online Education (AI-ALOE) which all have a focus on creating ethical and equitable AI learning environments.

Additionally, communities such as the Center for Integrative Research in Computing and Learning Sciences (CIRCLS) provide opportunities for these important conversations through events such as their recent virtual convening with multiple sessions that brought together stakeholders across fields. For example, there was an expertise exchange led by Dr. Diane Litman, Dr. Janice Gobert, and Aditi Mallavarapu on “Using the Learning Sciences and Computational Approaches to develop Assessments and Intelligent Tutoring Systems” that addressed many of the ideas and questions presented in this blog through discussions with practitioners in K12 and informal settings, higher education faculty, technology developers, researchers, and graduate students.

Through conversations across disciplines and with diverse stakeholders, we can take action to address core challenges with AI in education while also providing opportunities for learning analytics to shape educational settings in valuable ways.