EDM 2024 Speakers

Title: Generalizable and Interpretable Models of Learning

Abstract: Modeling learners’ knowledge, behavior, and strategies is at the heart of educational technology. Learner models serve as a basis for adapting the learning experience to students’ needs and supporting teachers in classroom orchestration. Consequently, a large body of research has focused on creating accurate models of student knowledge and behaviors. However, current modeling approaches are still limited: they are either defined for specific and well-structured domains (e.g., algebra, vocabulary learning) requiring substantial work from experts and limiting generalizability, or they lack interpretability. Recent advances in generative AI, in particular large language models (LLMs), have the potential to address these constraints. However, LLMs lack alignment with educational goals and a grounded knowledge. In this talk, I will discuss the key challenges in developing generalizable and explainable models, and our solutions to address them, including models tracking learning in open-ended environments and generalizing between different environments and populations. I will present our work on explainable AI, including a rigorous evaluation of existing approaches, the development of inherently interpretable models, as well as studies on effectively communicating model explanations. Finally, I will show some of our recent results combining “traditional” modeling approaches and LLMs to provide interpretable feedback and explanations while not compromising on model trustworthiness.

Title: Opportunities and Challenges for LLM Agent-Based Support for Collaborative Design

Abstract: Supporting collaborative design is an ideal context for exploring the capabilities and limitations of LLM-based conversational agents.  The ability to extract information in context and produce a coherent sounding text can be used to generate reflection triggers.  In two recent studies, we have employed LLM-based conversational agents with the goal of triggering human reflection and learning during collaborative software design.  As humans engage in collaborative design, they employ their own abilities to reason abstractly, to decompose problems, and apply principles productively.  Reflection is a valuable activity for promoting human learning in these settings. However, what humans are able to do in terms of abstraction and reasoning as part of their creative problem solving is precisely what is most difficult for LLM agents to do.  In contrast to claims of “super-human performance” in the media, in this talk we will explore the complementarity of human intelligence and Artificial Intelligence. We will begin with results of a classroom study where LLM-based conversational agent support for collaborative software development was successful in increasing student learning.  From there we will move on to argue in favor of a research agenda for exploiting the complementarity both in terms of applying AI capabilities to the betterment of human learning as well as inspiring further extension of technical capabilities from insights derived from observation of human reflection and learning in collaborative design.

---

Test of Time Award Talk

Title: Assessing Student Learning in Open Ended Learning Environments From Sequential to Multimodal Data Analysis

Abstract. From my early days as an AIED and EDM researcher, I have focused on understanding how students learn, especially in scenarios where they have to construct and apply their knowledge to problem-solving tasks. Collaborating with peers, we developed open-ended learning environments (OELEs) where K-12 students build scientific models and apply them to solve real-world problems. Challenges arise, as students have to navigate with multiple tools in the computer-environments. Some students overcome these challenges to become effective learners while others struggle to progress often applying suboptimal learning strategies. John Kinnebrew and I began analyzing learners’ activity logs to study these differences, resulting in the Differential Sequence Mining algorithm, which earned us the best paper award at EDM 2012. Expanding on this work, we developed the Contextualized Difference Mining method for understanding students’ learning behaviors, for which we are receiving this Test of Time award.

In my talk, I will review our work on Differential sequence mining, and explore its applications in understanding students’ cognitive and metacognitive learning behaviors. We have leveraged these insights to provide adaptive support, helping students’ progress in our Open-Ended Learning Environments (OELEs). Beyond this, we have employed other sequential representations, such as Markov Chains and Hidden Markov Models, to analyze students’ activities and behaviors in the context of their learning and problem-solving tasks. Beyond this, our OELEs have advanced to facilitate students’ integrated learning of science, computing, and engineering problem-solving, including collaborative efforts in computer-based and embodied learning scenarios. From these richer learning environments, I will share insights into our latest efforts involving multimodal data analysis, incorporating video, speech, and activity logs. Using vision-based deep learning models and large language models (LLMs), we integrate analyses across modalities, offering a comprehensive understanding of students’ collaborative learning and problem-solving activities. In conclusion, I will discuss the potential implications of our work on shaping future of learning in classrooms.

---

EDM data set award

Title: From Vision to Value: Eight years of the Open Learning Analytics Dataset

Abstract: The Open Learning Analytics Dataset (OULAD) has been used by researchers, educators, and data science enthusiasts for over 8 years with more than 400 citations recorded. The citations included academic users, educators, developers of educational software and many other users. The primary intention during the time of release was to develop a large enough dataset, which enables testing and comparison of methods designed to analyse student data containing student traces from the management system and their socio-demographic information. The OULAD was created as a spin-off of the project OUAnalyse, a project improving student retention via machine learning, which was selected in 2020 by UNESCO as one of the four best projects using AI in education. Being deployed with 200,000 students of The Open University, the system is one of the largest predictive learning analytics systems deployed globally.

Our talk will introduce The Open University learning context and OUAnalyse to provide the audience with insights into the data. Next, we will discuss some details about what it constitutes to make such a dataset open together with the structure of the dataset itself. We will share some use cases showing the variety of work and research that OULAD supported so far. Based on our experience, we will point out some of the most common issues and mistakes users face when using the dataset, which should help with future usage and ideas!