ABSTRACT

Large language models (LLMs) have emerged as a powerful tool in education, offering novel approaches to analyzing educational data and enhancing learning experiences. However, they also present new challenges, such as finding ways to effectively implement LLMs into educational environments to aid, rather than detract, students from learning and ensuring that no toxic or irrelevant content is presented to students. They also present a new host of ethical questions regarding the degree to which LLMs can go in aiding and providing feedback to students. This workshop aims to bring together researchers, educators, and practitioners to discuss these opportunities and challenges in leveraging LLMs for education.

Keywords

1. INTRODUCTION

Large Language Models (LLMs) have emerged as a powerful tool in Education Data Mining (EDM), offering novel approaches to analyzing educational data and enhancing learning experiences [1]. They provide novel opportunities in designing and augmenting learning environments, aiding students and educators in various aspects of the learning process, and helping researchers gain insight into a variety of domains within the field of education. LLMs have been increasingly integrated into learning platforms in a variety of ways to support students, teachers, and researchers. On the students’ side, they offer new possibilities for adaptive learning, personalized tutors, and real-time feedback [7, 6, 5]. On the teachers’ side, LLMs can generate interactive educational content, such as quizzes and simulations, that adapt to individual teaching preferences and hence assist teachers when generating lesson plans [8]. Moreover, LLMs serve as an effective tool for researchers in a variety of settings, including educational data generation and analysis [2].

However, LLMs also present new challenges, such as finding ways to effectively implement LLM-based technologies into educational environments to aid, rather than detract, students from learning and ensuring that no toxic or irrelevant content is presented to students. They also present a new host of ethical questions regarding the degree to which LLMs can go in aiding and providing feedback to students [4, 3].

The goal of this workshop is to bring together researchers, educators, and practitioners to discuss these opportunities and challenges in leveraging LLMs for education. We will investigate the above-mentioned issues along several topics of interests related, but not limited, to:

• Exploring the use of LLMs for adaptive learning and personalized feedback.
• Exploring the use of LLMs for knowledge tracing.
• Investigating ethical considerations when using LLMs for education.
• Assessing the effectiveness of LLMs for learning.
• Integrating LLMs into the teaching curriculum and online learning platforms.
• Ensuring fairness and equitable access to LLMs.
• Exploring the role of LLMs in collaborative learning environments.
• Investigating the impact of LLMs on student motivation and engagement.
• Examining the potential of LLMs for generating educational content and resources.
• Training educators to effectively utilize LLMs in a professional environment and fostering skills needed.
• Utilizing and fine-tuning open-source LLMs for usage in education.

2. ORGANIZERS

As organizers, we bring in the necessary expertise on topics relevant to this workshop. Furthermore, we have extensive experience organizing related workshops (including physical, hybrid, and virtual events). Our expertise and experience in topics related to LLMs for education would ensure that the proposed workshop is successful. Three of the organizers (Heffernan, Moore, and Singla) recently organized a workshop on a similar topic at NeurIPS’23, which received considerable interest from the educational community with about 150 participants and about 50 paper submissions – details are reported in [1]. Below, we provide a full list of organizers and program committee members.

• Neil Heffernan, nth@wpi.edu, Worcester Polytechnic Institute
• Rose E. Wang, rewang@stanford.edu, Stanford University
• Christopher MacLellan, cmaclellan3@gatech.edu, Georgia Tech
• Arto Hellas, arto.hellas@aalto.fi, Aalto University
• Chenglu Li, chenglu.li@utah.edu, University of Utah
• Joshua Littenberg-Tobias, joshua_tobias@wgbh.org, WGBH Education
• David Joyner, djoyner3@gatech.edu, Georgia Tech
• Steven Moore, stevenmo@andrew.cmu.edu, Carnegie Mellon Univsersity
• Adish Singla, adishs@mpi-sws.org, Max Planck Institute for Software Systems
• Zachary A. Pardos, pardos@berkeley.edu, University of California Berkeley
• Maciej Pankiewicz, mpank@upenn.edu, University of Pennsylvania
• Juho Kim, juhokim@kaist.ac.kr, KAIST
• Shashank Sonkar, ss164@rice.edu, Rice University
• Clayton Cohn, clayton.a.cohn@vanderbilt.edu, Vanderbilt University
• Anthony Botelho, abotelho@coe.ufl.edu, University of Florida
• Andrew Lan, andrewlan@cs.umass.edu, University of Massachusetts Amherst
• Lan Jiang, lanj3@illinois.edu, University of Illinois at Urbana-Champaign
• Mingyu Feng, mfeng@wested.org, WestEd
• Tanja Käser, tanja.kaeser@epfl.ch, EPFL
• Candace Walkington, cwalkington@mail.smu.edu, Southern Methodist University
• Eamon Worden, elworden@wpi.edu, Worcester Polytechnic Institute

3. WORKSHOP FORMAT

This will be a full-day workshop. The morning will be devoted to presentations and discussions of accepted papers. We will organize presenters into sessions addressing major themes (e.g., “Adaptive Learning”, “Effective Implementation”, “Ethical Considerations”). In total, we expect to have 3-4 themes addressed during the workshop related to LLMs for students, teachers, data analytics, and ethics. Each presenter will have 10 minutes to present, followed by 5 minutes for questions and discussions. At the end of each theme session, we will hold a short panel discussion, including the presenters, structured around the specific theme.

The afternoon will start with a poster session for all the accepted papers. Following the poster session, we will organize free-form discussions consisting of three sets of breakout sessions focusing on specific questions. Attendees can choose which session they wish to attend. Session A will focus more on implementing LLMs into online learning platforms for a variety of student-related purposes, including personalized learning assistants, feedback automation, and chatbots; Session B will focus on using LLMs for teacher-related purposes such as knowledge tracing, content generation, and automated grading; Session C will focus on the ethics of using LLMs, ensuring fair and equal access to LLMs, and considerations regarding open- and closed-source LLMs. More concretely, we plan to address the following questions in these breakout sessions:

• How can we most effectively utilize LLMs in learning environments? What sort of supports are most effective for student learning? How are these supports best implemented?
• How can we engage students with LLMs in an effective manner? What do students want out of LLMs? What should LLMs provide students to aid, not detract, from their learning?
• How personalized can LLMs be when they know students worse than the teacher? How can we determine the effectiveness of LLMs in student learning? How do we best scale the usage of LLMs in platforms?
• How can LLMs best aid teachers? How can LLMs best generate content? Can LLMs effectively generate personalized content for teachers?
• How can LLMs effectively report student performance to teachers? How can they be used in knowledge tracing and similar analysis? Can these reports scale to entire classrooms?
• How do we convince teachers that LLMs can be beneficial? What are teacher’s concerns about LLMs? How can we alleviate their concerns and implement LLMs in an effective way?
• What are the ethical considerations when using LLMs in education? What are the concerns when providing LLM-generated content to students? What are the concerns when using LLMs for other purposes, such as teacher reports or question generation?
• What potential ways might LLMs harm students? What safety measures need to be taken to minimize the harm from LLMs? Can we determine whether LLM-generated content is harmful?
• How do we ensure equal access to LLMs regardless of social standing or other factors? How can we address potential biases in LLMs to ensure fair and equitable outcomes for all users? What are the concerns about using closed-source and open-source LLMs?

Given that the workshop themes are very timely and of huge significance to the EDM community, we expect the workshop will attract considerable interest and engagement. In total, we expect 75-100 participants in the workshop with a range of skills, expertise, and backgrounds.

4. POST-WORKSHOP PLANS

The workshop program is designed to facilitate new connections and create fruitful partnerships. After the workshop, we will provide online networking and community-building opportunities among the workshop participants, e.g., through a dedicated mailing list or blogs. We will also summarize the findings of the workshop in the form of a technical report that could be useful for the research community.

5. CALL FOR PARTICIPATION

This workshop will explore the various usages, implementations, and theories supporting the deployment of LLMs into online education. Further, we will examine the dangers, ethical considerations, and potential solutions related to the use of LLMs in education, ensuring a comprehensive understanding of their impact on learning environments. We also intend to explore the best ways to analyze the performance of LLMs, and use LLMs as tools for analysis. We invite papers (of 4 to 8 pages in EDM Proceedings format, not including references) addressing issues related to using LLMs in educational contexts and learning engineering platforms. The key issues and topics of interest for the workshop are provided in Section 1.

We will make the call more welcoming by highlighting that the workshop welcomes attendance from individuals who do not have something they’d like to submit but are still interested in the workshop topics. Based on our previous experience in organizing similar workshops, we expect to receive over 30 submissions and will accept around 20 papers.

6. ADDITIONAL AUTHORS

Additional authors: Joshua Littenberg-Tobias (WGBH Education, joshua_tobias@wgbh.org) and David Joyner (Georgia Tech, djoyner3@gatech.edu) and Steven Moore (Carnegie Mellon University, stevenmo@andrew.cmu.edu) and Adish Singla (Max Planck Institute for Software Systems, adishs@mpi-sws.org) and Zach A. Pardos (University of California Berkeley, pardos@berkeley.edu) and Maciej Pankiewicz (University of Pennsylvania, mpank@upenn.edu) and Juho Kim (KAIST, juhokim@kaist.ac.kr) and Shashank Sonkar (Rice University, ss164@rice.edu) and Clayton Cohn (Vanderbilt University, clayton.a.cohn@vanderbilt.edu) and Anthony Botelho (University of Florida, abotelho@coe.ufl.edu) and Andrew Lan (University of Massachusetts Amherst, andrewlan@cs.umass.edu) and Lan Jiang (University of Illinois at Urbana-Champaign, lanj3@illinois.edu) and Mingyu Feng (WestEd, mfeng@wested.org) and Tanja K"aser (EPFL, tanja.kaeser@epfl.ch) and Anna Rafferty (Carleton College, arafferty@carleton.edu) and Eamon Worden (Worcester Polytechnic Institute, elworden@wpi.edu)

7. REFERENCES

1. P. Denny, S. Gulwani, N. T. Heffernan, T. Käser, S. Moore, A. N. Rafferty, and A. Singla. Generative AI for Education (GAIED): Advances, Opportunities, and Challenges. CoRR, abs/2402.01580, 2024.
2. D. Hirunyasiri, D. R. Thomas, J. Lin, K. R. Koedinger, and V. Aleven. Comparative Analysis of GPT-4 and Human Graders in Evaluating Praise Given to Students in Synthetic Dialogues. CoRR, abs/2307.02018, 2023.
3. E. Kasneci et al. ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education. Learning and Individual Differences, 103, 2023.
4. M. Lee et al. Evaluating Human-Language Model Interaction. CoRR, abs/2212.09746, 2022.
5. H. A. Nguyen, H. Stec, X. Hou, S. Di, and B. M. McLaren. Evaluating ChatGPT’s Decimal Skills and Feedback Generation in a Digital Learning Game. In European Conference on Technology Enhanced Learning, pages 278–293. Springer, 2023.
6. M. Pankiewicz and R. S. Baker. Large Language Models (GPT) for Automating Feedback on Programming Assignments. In International Conference on Computers in Education Conference Proceedings, Volume I, pages 68–77, 2023.
7. Z. A. Pardos and S. Bhandari. Learning gain differences between chatgpt and human tutor generated algebra hints. arXiv preprint arXiv:2302.06871, 2023.
8. Z. Wang, J. Valdez, D. Basu Mallick, and R. G. Baraniuk. Towards Human-like Educational Question Generation with Large Language Models. In International Conference on Artificial Intelligence in Education, pages 153–166. Springer, 2022.