ABSTRACT

The computer-based learning environment (CBLE) is designed for instructional purposes and to support the learner to understand challenging and complex topics that are difficult to describe or comprehend. In CBLE, learners can access information in various formats such as text, diagrams, graphs, audio, video, etc. to learn. To successfully interact and learn from CBLE, the learners should plan their learning strategy, identify all the learning paths to achieve their learning goal, and select the most suitable one. However, navigating in such an environment can overwhelm learners’ working memory, leading to cognitive overload, and disorientation which makes hurdles in learning. Several empirical studies have investigated overcoming the above challenges. They have reported, that the learners should be provided with metacognitive support. Metacognition is one of the strategies for encouraging self-regulated learning (SRL) in CBLEs. Hence, in our research, we propose to provide metacognitive prompts to learners while they interact with the CBLE and analyze the impact of metacognitive prompts.

Keywords

INTRODUCTION

The Computer-Based Learning Environment (CBLE) aims to support learners in achieving their objectives across a range of disciplines [1]. It incorporates multimedia, text, images, animations, simulations, audio, and video representations, among other things [6] for learners to access information [5]. Although CBLE provides excellent resources, it can also present challenges for learners. Since these environments give learners a high degree of control [6], they can follow their instructional path and access numerous representations of information as well as opportunities to manipulate them [5]. However, managing such an interactive and complex system actively can overwhelm learners' working memory, leading to cognitive overload, disorientation, and impeding the learning process [5]. Moreover, it has been reported that to acquire conceptual knowledge of a complex topic, learners should be able to constantly identify relevant information, track progress toward the goal, and sub-goals, and make judgments about their learning as per their learning progress [6] [9].

Recent studies have found that most learners are unable to manage their learning and they struggle to regulate multiple learning processes and as a result, learn less conceptually [1]. Students who can self-regulate their learning effectively are likely to acquire a conceptual understanding of complex topics [6]. Several empirical studies have investigated that to overcome the challenges provided by the environment, it is necessary to use metacognitive skills like monitoring, planning, and reflecting [5]. In order to engage in the planning, strategy usage, and monitoring processes, learners who do not self-initiate effective SRL processes should be assisted in identifying the metacognitive processes that are most effective for them [6]. Metacognitive support is one strategy for encouraging self-regulated learning [4]. The use of prompting as an instructional strategy is becoming more popular, particularly in the area of computer-based learning environments where prompting is simple to implement [6]. Several studies have revealed that metacognitive prompts direct the learners’ awareness and monitor their learning activity which led to improvement in the planning, monitoring, and reflection activities in addition to learning outcomes [2] [7] [8]. In our study, we are intended to investigate the impact of metacognitive prompts on learning gain in CBLE. And also investigate possible factors that may have influenced the effectiveness of metacognitive prompts.

Research Questions

The focus of this study lies on metacognitive prompts, a topic that has been extensively investigated in the literature [3] [1] [4]. Drawing upon prior research metacognitive prompts can be categorized based on aspects such as modality, adaptability, and specificity. Mode of delivery is one such aspect, with prompts being classified as on-screen text, pop-up windows, virtual images, and auditory narration. Additionally, prompts can be tailored to the task at hand or learning situation, with adaptive prompts tailored to the individual needs of each student, while fixed prompts remain the same for all students. The effects of these prompts on metacognitive strategies and learning outcomes have been found to vary depending on the moderator variable [7]. However, most studies in this area have been conducted in the fields of social science (e.g., education, psychology) and science (e.g., math, biology, physics). Fewer studies have been conducted in the domain of engineering and technology, and even fewer have focused on problem-solving learning. While several studies have examined the impact of personalized metacognitive prompts and feedback on learning performance, there is still insufficient data on the performance of transfer and retention tasks, which would provide a clearer picture of the long-term effects of these prompts. Therefore, further research is needed to address some of the key research questions outlined below.

1. Do domain-specific, personalized metacognitive prompts with feedback help in enhancing the performance of transfer and retention tasks?
2. Do domain-specific, personalized metacognitive prompts with feedback help in enhancing metacognitive strategies?

Proposed Contribution

To achieve our research goals, we propose to implement Design-based research (DBR) for the design and development of a CBLE aimed at promoting metacognition and improving learning performance among undergraduate engineering learners. Our study aims to investigate the impact of metacognitive prompts on learning gains in the CBLE and factors that may influence their effectiveness. The CBLE environment will feature the integration of concepts, practices, videos, text, simulations, and personalized prompts with feedback. A tentative plan outlining the research tasks to be undertaken is presented for further exploration.

Step 1 - Literature review in the context of undergraduate engineering classroom

The primary aim of this literature review was to examine the different interventions used to foster metacognition and different methodologies were used to measure the impact of the intervention on student performance. We identified primarily three intervention methods that were used to foster metacognition and train the purpose and strategies of metacognition like workshops, reading materials, and rubrics to guide learners. However, we identified three different methods to measure metacognitive awareness, reflection journals on their learning, metacognitive awareness questionnaires, and semi-structured interviews.

Step 2- Conduct a study to identify and assess the metacognitive awareness of undergraduate engineering learners

On the basis of the literature review, we conducted a research study with engineering students to understand the metacognitive process. We found that students mostly use control and regulation while using an open-ended learning environment. The low-scoring students often don’t perform monitoring and reflection phases.

Step 3- Design and develop a system to foster engineering learners’ metacognition

The proposed CBLE system is designed to help learners learn about electrical circuits. This CBLE environment will integrate concepts, practices, videos, text, simulations, and personalized prompts with feedback. The simulator will be designed to be user-friendly, with a variety of tools and features to help students build and analyze circuits. In addition to the simulator, the system will include a variety of video content that covers the key concepts and principles of electrical circuits. The videos will also provide step-by-step instructions on how to use the circuit simulator, so students can quickly get up to speed. It will also include text content that covers the same topics as the videos. The text content will be designed to be comprehensive and easy to understand, with clear explanations and examples. The text content will be organized into a list of topics to learn, so students can navigate and find the necessary information. The system will monitor the learner's progress and performance within the CBLE, and provide personalized metacognitive prompts to the learner when they are struggling or when they have made a mistake. Figure 1 shows our proposed study design and expected outcome

Step 4-Design the study to collect data and then analyze the data

To address the research questions, we are targeting undergraduate students. Figure 2 shows the study design and data collection. The proposed CBLE system will have two versions, the Experimental Group (EG) will use the CBLE system with personalized metacognitive prompts and feedback, while the Control Group (CG) will use a CBLE system without personalized prompts and feedback. Participants will be randomly assigned to either experimental or control groups.

Data Collection: We plan to collect both groups' log data like learners’ activity logs which track the actions taken by learners within the CBLE system, such as viewing content, completing exercises, and interacting with the simulation tools. And the time spent on each activity. This log data can provide insights into how learners engage with the system. Log data will be used to track the use of metacognitive prompts, these logs will provide insight into when and how often prompts are being used by students. Along with logs we have planned to collect pre-test, and post-test scores, and learners’ reflections to analyze learners’ performance. The pre-and post-tests will consist of multiple-choice questions and open-ended questions that assess students' understanding of the concepts covered in the CBLE system.

A delayed post-test will be administered a few weeks or months after the completion of the course to measure retention of learning. A transfer task can be administered to measure the extent to which students can apply what they have learned in a new context.

To analyze the learning strategies, the Motivated Strategies for Learning Questionnaire (MSLQ), a metacognitive questionnaire will be used. MSLQ can provide insights into the impact of personalized metacognitive prompts and feedback on students' learning strategies data and learners’ reflections. Along with this, we will collect qualitative data through interviews.

Data Analysis: Data will be analyzed using both descriptive and inferential statistics. Descriptive statistics will be used to summarize the data and to identify any patterns or trends. Inferential statistics will be used to determine whether there are significant differences between the experimental and control groups in terms of student performance.

cONCLUSION

In conclusion, this research paper proposed the use of Design-based research (DBR) to design and develop a Computer-Based Learning Environment (CBLE) aimed at promoting metacognition and improving learning performance among undergraduate engineering learners. The study aimed to investigate the impact of metacognitive prompts on learning gains in the CBLE and factors that may influence their effectiveness. Based on a literature review, a study was conducted to identify and assess the metacognitive awareness of undergraduate engineering learners. A CBLE system was designed and developed, integrating concepts, practices, videos, text, simulations, and personalized prompts with feedback. A study design was proposed to collect data, including pre-tests, post-tests, and delayed post-tests, along with qualitative data through interviews. Data analysis will be conducted using both descriptive and inferential statistics. The expected outcomes of this research are to contribute to the understanding of the impact of metacognitive prompts on learning gains in a CBLE and factors that may influence their effectiveness. This research has implications for the design and development of effective CBLEs that can promote metacognition and improve learning outcomes.

REFERENCES

1. Azevedo, R., Cromley, J. G., & Seibert, D. (2004). Does adaptive scaffolding facilitate students’ ability to regulate their learning with hypermedia?. Contemporary educational psychology, 29(3), 344-370.
2. Azevedo, R., Cromley, J. G., Moos, D. C., Greene, J. A., & Winters, F. I. (2011). Adaptive content and process scaffolding: A key to facilitating students' self-regulated learning with hypermedia. Psychological Test and Assessment Modeling, 53(1), 106.
3. Bannert, M., Hildebrand, M., & Mengelkamp, C. (2009). Effects of a metacognitive support device in learning environments. Computers in human behavior, 25(4), 829-835.
4. Engelmann, K., Bannert, M., & Melzner, N. (2021). Do self-created metacognitive prompts promote short-and long-term effects in computer-based learning environments?. Research and Practice in Technology Enhanced Learning, 16(1), 1-21.
5. Winters, F. I., Greene, J. A., & Costich, C. M. (2008). Self-regulation of learning within computer-based learning environments: A critical analysis. Educational psychology review, 20, 429-444.
6. Greene, J., Moos, D., & Azevedo, R. (2011). Self-regulation of learning with computer-based learning environments. New directions for teaching and learning. Publicado en línea en Wiley Online Library. https://doi. org/10.1002/tl, 449.
7. Guo, L. (2022). Using metacognitive prompts to enhance self-regulated learning and learning outcomes: A meta-analysis of experimental studies in computer-based learning environments. Journal of Computer Assisted Learning, 38(3), 811-832.
8. Pieger, E., & Bannert, M. (2018). Differential effects of students' self-directed metacognitive prompts. Computers in Human Behavior, 86, 165-173.
9. Azevedo, R. (Ed.). (2018). Computers as Metacognitive Tools for Enhancing Learning: A Special Issue of Educational Psychologist. Routledge.

© 2023 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.