Chapter 27

Handbook of Learning Analytics
First Edition

A Critical Perspective on Learning
Analytics and Educational Data Mining

Rita Kop, Helene Fournier & Guillaume Durand


In our last paper on educational data mining (EDM) and learning analytics (LA; Fournier, Kop & Durand, 2014), we concluded that publications about the usefulness of quantitative and qualitative analysis tools were not yet available and that further research would be helpful to clarify if they might help learners on their self-directed learning journey. Some of these publications have now materialized; however, replicating some of the research described met with disappointing results. In this chapter, we take a critical stance on the validity of EDM and LA for measuring and claiming results in educational and learning settings. We will also report on how EDM might be used to show the fallacies of empirical models of learning. Other dimensions that will be explored are the human factors in learning and their relation to EDM and LA, and the ethics of using “Big Data” in research in open learning environments.

Export Citation: Plain Text (APA)     BIBTeX     RIS

Supplementary Material
No Supplementary Material Available
References (40)

Atkinson, S. P. (2015). Adaptive learning and learning analytics: A new learning design paradigm. BPP Working paper.

Bates, T. (2014). Two design models for online collaborative learning: Same or different?

Berland, M., Baker, R. S., & Blikstein, P. (2014). Educational data mining and learning analytics: Applications to constructionist research. Technology, Knowledge and Learning, 19, 206–220.

Bista, K. (2014). Is Twitter an effective pedagogical tool in higher education? Perspectives of education graduate students. Journal of the Scholarship of Teaching and Learning, 15(2), 83–102.

Bouchard, P. (2013). Education without a distance: Networked learning. In T. Nesbit, S. M. Brigham, & N. Taber (Eds.), Building on critical traditions: Adult education and learning in Canada. Toronto: Thompson Educational Publishing.

Boyd, D. (2010). Privacy and publicity in the context of big data. Paper presented at the 19th International Conference on World Wide Web (WWW2010), 29 April 2010, Raleigh, North Carolina, USA.

Boyd, D., & Crawford, K. (2012). Critical equations for Big Data. Information, Communication & Society, 15(5), 662–679. doi:10.1080/1369118X.2012.678878

British Science Council. (2009). What is science?

Cavoukian, A., & Jonas, J. (2012). Privacy by design in the age of big data.

Cormack, A. (2015). A data protection framework for learning analytics.

Christopher, J. C., Wendt, D. C., Marecek, J., & Goodman, D. M. (2014) Critical cultural awareness: Contributions to a globalizing psychology. American Psychologist, 69, 645–655.

Denzin, N., & Lincoln, Y. (Eds.) (2011). The Sage handbook of qualitative research (4th ed.). Thousand Oaks, CA: Sage.

Durand, G., Belacel, N., & Goutte, C. (2015) Evaluation of expert-based Q-matrices predictive quality in matrix factorization models. Proceedings of the 10th European Conference on Technology Enhanced Learning (EC-TEL ʼ15), 15–17 September 2015, Toledo, Spain (pp. 56–69). Springer. doi:10.1007%2F978-3-319-24258-3_5
Ed Tech Review (2016). Educational Data Mining (EDM).

El Emam, K. (1998). The predictive validity criterion for evaluating binary classifiers. Proceedings of the 5th International Software Metrics Symposium (Metrics 1998), 20–21 November 1998, Bethesda, MD, USA (pp. 235–244). IEEE Computer Society.

European Data Protection Supervisor. (2015). Leading by example: The EDPS strategy 2015–2019.

Fenwick, T. (2015a). Things that matter in education. In B. Williamson (Ed.), Coding/learning, software and digital data in education. University of Stirling, UK.

Fenwick, T. (2015b). Professional responsibility in a future of data analytics. In B. Williamson (Ed.) Coding/learning, software and digital data in education. University of Stirling, UK.

Fournier, H., & Kop, R. (2015). MOOC learning experience design: Issues and challenges. International Journal on E-Learning, 14(3), 289–304.

Gergen, J. K., Josselson, R., & Freeman, M. (2015). The promise of qualitative inquiry. American Psychologist, 70(1), 1–9.

Gonzalez-Brenes, J., & Huang, Y. (2015). Your model is predictive — but is it useful? Theoretical and empirical considerations of a new paradigm for adaptive tutoring evaluation. In O. C. Santos, J. G. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J. M. Luna, C. Mihaescu, P. Moreno, A. Hershkovitz, S. Ventura, & M. Desmarais (Eds.), Proceedings of the 8th International Conference on Education Data Mining (EDM2015), 26–29 June 2015, Madrid, Spain (pp. 187–194). International Educational Data Mining Society.

Gupta, A. (2013). Fraud and misconduct in clinical research: A concern. Perspectives in Clinical Research. 4(2), 144–147. doi:10.4103/2229-3485.111800.

Hardt, M. (2014). How big data is unfair: Understanding sources of unfairness in data driven decision making.

Jones, C., Dirckinck-Holmfeld, L., & Lindström, B. (2006). A relational, indirect, meso-level approach to CSCL design in the next decade. International Journal of Computer Supported Collaborative Learning, 1(1), 35–56.

Kansanen, P., & Meri, M. (1999). The didactic relation in the teaching–studying–learning process. TNTEE Publications, 2, 107–116.

Kitchin, R. (2015). Foreword: Education in code/space. In B. Williamson (Ed.), Coding/learning, software and digital data in education. University of Stirling, UK.

Kop, R. (2012). The unexpected connection: Serendipity and human mediation in networked learning. Educational Technology & Society, 15(2), 2–11.

Kop, R., Fournier, H., & Durand, G. (2014). Challenges to research in massive open online courses. Merlot Journal of Online Learning and Teaching, 10(1).

Long, Y., & Aleven, V. (2014). Gamification of joint student/system control over problem selection in a linear equation tutor. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. Panourgia (Eds.), Proceedings of the 12th International Conference on Intelligent Tutoring Systems (ITS 2014), 5–9 June 2014, Honolulu, HI, USA (pp. 378–387). New York: Springer. doi:10.1007/978-3-319-07221-0_47

Long, C., & Siemens, G. (2011, September 12). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5).

Meyer, J. H. F., & Land, R. (2006). Overcoming barriers to student understanding: Threshold concepts and troublesome knowledge. London: Routledge.

Mischel, W. (1968). Personality and assessment. London: Wiley.

Oblinger, D. G. (2012, November 1). Analytics: What we’re hearing. EDUCAUSE Review.

Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). Population validity for educational data mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487–501.

Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438–450.

Siemens, G., Dawson, D., & Lynch, L. (2013). Improving the quality and productivity of the higher education sector: Policy and strategy for systems-level deployment of learning analytics. SoLAR.

Stewart, B. (2015). Open to influence: What counts as academic influence in scholarly networked Twitter participation. Learning, Media and Technology, 40(3), 287–309. doi:10.1080/17439884.2015.1015547

Weller, M. (2011). A pedagogy of abundance. Spanish Journal of Pedagogy, 249, 223–236.

Wen, M., Yang, D., & Rosé, C. (2014). Sentiment analysis in MOOC discussion forums: What does it tell us? In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (EDM2014), 4–7 July, London, UK (pp. 130–137). International Educational Data Mining Society.

Xu, D., & Smith Jaggars, S. (2013). Adaptability to online learning: Differences across types of students and academic subject areas. CCRC Working Paper No. 54. Community College Research Center, Teachers College, Columbia University.

About this Chapter

A Critical Perspective on Learning Analytics and Educational Data Mining

Book Title
Handbook of Learning Analytics

pp. 319-326




Society for Learning Analytics Research

Rita Kop1
Helene Fournier2
Guillaume Durand2

Author Affiliations
1. Faculty of Education, Yorkville University, United Kingdom
2. Information and Communications Technologies, National Research Council of Canada, Canada

Charles Lang3
George Siemens4
Alyssa Wise5
Dragan Gašević6

Editor Affiliations
3. Teachers College, Columbia University, USA
4. LINK Research Lab, University of Texas at Arlington, USA
5. Learning Analytics Research Network, New York University, USA
6. Schools of Education and Informatics, University of Edinburgh, UK

Founding Members
Previous Image
Next Image

info heading

info content