ABSTRACT

In an era of digital media and hyperconnectivity, individuals are frequently flooded with an abundance of information, much of which they are unable to effectively process and are thus vulnerable to misinformation. One particularly harmful form of misinformation is the proliferation of "fake news," which can have a damaging effect on the social cohesion of communities. In response to this issue, educational practitioners in various nations are striving to empower learners with the ability to identify and refute such misinformation. The present author is also contributing to this effort through the development of a technology-enhanced learning environment that is intended to foster critical thinking in learners.

Keywords

INTRODUCTION

In contemporary society, individuals are frequently overloaded with an abundance of information. However, this flood of information can often prove overwhelming, resulting in a vulnerability to misinformation. The internet serves as the primary means through which individuals access information. However, the knowledge readily available through this medium possesses distinct characteristics as compared to that traditionally provided by educators and educational texts [6]. Internet search results often comprise multiple accounts with varying scopes, arguments, and levels of support. Furthermore, online sources may vastly differ in terms of authorship, purpose, perspective, legitimacy, and justification techniques.

As previously discussed, the proliferation of misinformation in contemporary society has resulted in an increased risk of the formation of false beliefs among citizens. The inability to differentiate between legitimate and illegitimate information can lead to the acceptance of both as factual. To combat this, citizens must possess specialized skills that enable them to effectively navigate and evaluate the credibility and reliability of the vast amount of information available online. In light of this, there is a pressing need to understand and foster critical data literacy within the fields of educational research and practice. The objective of the present research is to facilitate the development of these skills among students, enabling them to proficiently analyze and scrutinize the reliability of complex online information.

Background

In this background section, the author will first attempt to arrive at a working definition of critical thinking. In the next subsection, a number of organizations working to combat misinformation will be discussed. This will be followed by a discussion of academic research on misinformation and critical thinking in the next subsection. Finally, the author will discuss the current status of work in this area and how their own proposed research study will add to the existing knowledge.

Critical Thinking

Dwyer, Hogan, & Stewart define critical thinking as “a metacognitive process that, through purposeful, reflective judgment, increases the chances of producing a logical conclusion to an argument or solution to a problem” [7]. For the purpose of this paper, the author will use this definition as a working definition. However, it is necessary to also understand the broader meaning of critical thinking. To start with, Ennis has outlined abilities such as analyzing arguments, claims, or evidence, making inferences using inductive or deductive reasoning, judging or evaluating, and making decisions or solving problems as essential parts of critical thinking [8]. He has further identified behaviors relevant to critical thinking such as asking and answering questions for clarification; defining terms; identifying assumptions. Thus, critical thinking consists of a cluster of skills and behavior to analyze complex information. Looking at these aspects of critical thinking, it seems to be an effective tool for combating misinformation. However, fighting misinformation at an individual level is not enough. Fortunately, several organizations are also currently working on fighting misinformation, the details of their work are covered in the next section.

Misinformation Bunking Initiatives

Top universities and SMEs from seven different European nations are partners in the EU-funded initiative Co-Inform. The goal is to develop tools that promote digital literacy and critical thinking for a more informed society [5]. Their objective is to give individuals, journalists, and politicians the resources they need to recognize "fake news" online, comprehend how it spreads, and access reliable information. Co-Inform offers two main tools to combat misinformation. First is a browser plugin to increase citizens' awareness of content that is entirely or partially inaccurate, relevant fact-checking articles and remedial information, how ordinary citizens see this content, and important comments from fellow citizens that are both in favor of and against it. Second is a dashboard for fact-checking journalists and policymakers that displays discovered misinformation, its source, how and where it spreads and will spread in the future, the public's impression of it now and in the future, and the most important comments made by the public.

There are also dedicated websites both at the global level and in India to track and debunk fake news and misinformation. Snopes, formerly known as the Urban Legends Reference Pages, is one such fact-checking website [20]. Snopes seeks to disprove or validate widely circulated urban legends. Similarly, Alt News is a fact-checking website based out of India that works to dispel the falsehoods, lies, and misinformation that people often come across in both mainstream and social media [2]. Politics, social media rumors, mainstream media misinformation, and bias are just a few examples of the inaccurate information that Alt News fact-checks. Apart from these initiatives for combating misinformation, there has also been a fair amount of academic research conducted in this field, the next section elaborates on that.

Digital Literacy Research

Digital literacy has been defined as the ability and knowledge required to effectively navigate the complex and fragmented world of information available online. In simpler terms, it means having the skills needed to find, understand, and use information on the internet. [9]. Three types of digital skills have been identified by Ng: technological (using technology tools); cognitive (using critical thinking while handling information); and social (communicating and socializing) [18]. In the context of the educational process, thinking skills have been recognized as an important component of digital literacy, along with technical abilities [13]. Sulzer asserts that digital thinking will include identifying misinformation, echo chambers, and fake news [21]. Thus, it can be argued that digital thinking is the term used by digital literacy practitioners to refer to critical thinking while engaging with information online.

Students engage with online information through their personal epistemology. This ability is related to their perspectives about knowledge and knowing. Kuhn and Park have characterized epistemological understanding at four levels [16]. At the first level, realists consider knowledge to be definite and to emanate from an outside source. They think it's not vital to use critical thinking. At the second level, absolutists think knowledge is certain and emanates from a distant source that is inaccessible. Thus, critical thinking serves as a means for people to evaluate claims in light of reality and decide whether they are true or not. Multiplists, who consider knowledge to be created by human minds and hence uncertain, are found at the third level. As a result, they believe that critical thinking is useless. The fourth level is where evaluativists reside, who think that knowledge is created by human minds, and is unclear, yet subject to review. They consider critical thinking as a tool for supporting reasonable claims and advancing understanding. In order to better understand the relationship between students' individual epistemologies and their online learning practices, Barzilai and Zohar studied 38 sixth-graders [4]. The results demonstrate the significance of epistemic thinking in online inquiry learning. Students who were more familiar with evaluation strategies and criteria performed more frequent and thorough website evaluations. Students who were more conscious of the potential for discrepancies between online accounts and the necessity of constructing knowledge by integrating several viewpoints were more likely to identify discrepancies between the points of view of various websites, compare them, and build an argument based on a variety of online sources. Even though personal epistemology can account for a fair portion of the processes people use to process information, it also involves other factors like cognitive biases and epistemic emotions. Fig 1 explains the various factors involved in this process.

Different research groups over the years have tried to design strategies to teach students how to use critical thinking to spot fake news. The Association of College and Research Libraries (ACRL) introduced its Framework for Information Literacy for Higher Education in 2015, which was officially adopted by the ACRL Board in January of the following year. This framework emphasizes the importance of learners utilizing research tools and evaluating the credibility of sources in order to develop their information literacy skills [3]. One tool that can assist students in this process is the use of LibGuides, which are web-based applications that allow for the creation and organization of electronic guides. These can be easily embedded in course and library websites and accessed by students online. An example of a useful LibGuide in this context is the "Fake" News guide created by librarian Eric Novotny in 2017 at The Pennsylvania State University. The guide covers various forms of fake news, such as satire, bias, and clickbait. Another helpful tool is the use of worksheets, such as the CRAAP (Currency, Relevance, Authority, Accuracy, and Purpose) worksheet developed by California State University, Chico in 2010. In contrast, The Global Digital Citizen Foundation (2015) promotes a different approach to critical thinking by using a "Who, What, Where, When, Why, and How" method.

In spite of the above efforts, a recent intervention study focused on teaching students to evaluate search results and select websites to open revealed that students frequently resorted to less effective tactics when analyzing results, using their familiarity with a website and its top-level domain to determine its reliability, despite teachers' best efforts to teach them strategies for evaluating results modeled on fact checkers’ approaches [17]. This means that just informing students about strategies is not enough. Critical thinking is not just a bunch of skills but also an attitude which can only be inculcated through practice. A better strategy would be to provide opportunities to students to practice critical thinking abilities in context of actual problems in a learning environment. In this environment, they can practice their critical thinking abilities for long durations and hone them over time with proper support. Agesilaou & Kyza designed and implemented a Learning Environment to foster critical data literacy [1]. Their work describes the design-based research process of designing an educational intervention to foster critical data literacy through the use of self-tracking devices. While their work focused on the issues of privacy and digital data, the author of this current paper plans to design a learning environment to foster critical thinking skills among students in order to specifically deal with misinformation present on the internet. The research goal will be to help students to develop effective critical thinking strategies to deal dealing with complex online information.

This background section helped define critical thinking as a metacognitive process that involves analyzing arguments, claims, and evidence, making inferences, solving problems and asking questions for clarification and identifying assumptions. It also helped to understand the work being done in both academic and non-academic spheres to counter the spread of misinformation online. It explained how the information transfer techniques used by teachers fall short in equipping students with critical thinking abilities and established the need for a learning environment which offers prolonged opportunities for students to practice and hone their critical thinking skills on authentic scenarios where they are also provided continuous support in terms of active scaffolds. In the next section, the author will elaborate on their proposed plan for the learning environment.

Proposed Solution

In order to build a coherent and credible learning environment to foster critical thinking, the author needs to use some theoretical perspectives to provide the foundation for such an intervention. One prominent theory is the dual processing theory, which proposes that individuals use two primary modes of thinking - System 1, which is intuitive and automatic, and System 2, which is more analytical and deliberate [10]. This theory is particularly relevant for this intervention design because it has been observed that many a times users fall prey to misinformation because they have not spent enough time on a piece of information and respond too quickly [11]. The author is aware that dual processing theory has faced criticism in recent years. However, there is still substantial evidence in cognitive science to support the dual-processing distinction [12]. The theory remains valid and useful in understanding the interplay between automatic and deliberate thinking processes.

In order to provide an authentic learning experience to the students, a problem-based learning (PBL) approach can be used, where students work in pairs to develop solutions to real-world problems related to misinformation [14]. In order to ensure a smooth collaboration between the student pairs, collaboration scripts will be used. Collaboration scripts are instructional tools that guide learners on how to interact with each other during learning activities [15]. They provide a sequence of learning activities and roles for learners to follow in order to promote collaborative learning. In this learning environment, the author plans to use a type of driver-navigator script where one partner searches for the information on the system while the other person guides them. The main motive behind this peculiar pairing is that it will require the learners to discuss and debate the entire time while they are searching for credible information because they will have their biases and beliefs.

A theory that can be leveraged to better understand this social interaction is the social judgment theory, which suggests that people's attitudes and beliefs are influenced by their perception of what others think [19]. By encouraging students to engage in discussions and debates with their peers, the learning environment can foster critical evaluation of information by considering multiple perspectives and identifying potential biases. Introducing the social learning aspect is particularly important for the students to learn to argue logically and identify fallacies and biases in other’s and their own opinions.

Also, as discussed in the background section above, different learners tend to use different ways of engaging with information online. There is also this idea of the behavioral pattern displayed by professional fact-checkers when trying to determine the authenticity of a piece of online media. The best way to measure these various techniques will be to use learning analytics to capture the interaction of learners with the learning environment using log data. Primarily, two kinds of interactions will be captured, first will be the frequency of user interaction with various com-ponents in the environment, and second will be the duration of those interactions. This will help the researcher categorize the various patterns, for example, one cohort of users might be clicking on a number of resources and spending little time on each of them while another cohort might be accessing only few resources but spend significant time on each resource. Later, the various cohorts would be analyzed in relation to their demonstration of critical thinking behaviors. In the next sub-section, the author will provide a sample learning task that they might use in their learning environment.

Sample Learning Problem

This scenario is designed to help students practice critical thinking skills in the context of evaluating claims made about a dietary supplement marketed as a weight loss aid. Two students as a pair will work to evaluate the claims made about the supplement and pronounce their verdict on whether it is a weight loss solution or not. They will also be asked to back up their verdict with proper evidence. The students will be given the following sources of information:

1. An advertisement that claims the supplement is a "miracle weight loss solution" and features testimonials from people who have lost weight while taking the supplement.
2. A medical study that reports on the potential health risks associated with the supplement, like organ damage and other serious side effects.
3. A warning from a (government) health agency that advises consumers to avoid the supplement due to its potential health risks.

To help students evaluate the information provided, they will be given the following questions:

1. What are the claims being made about the effectiveness of the dietary supplement as a weight loss aid?
2. What evidence is provided to support these claims, and how strong is this evidence?
3. What are the potential health risks associated with the supplement, and how serious are these risks?
4. What are the recommendations of (government) health agencies with regards to the supplement?
5. What are the potential biases or conflicts of interest that may be present in each source of information?

This scenario is designed to help students develop critical thinking skills related to evaluating the claims made about dietary supplements and to identify potential biases or conflicts of interest in the sources of information provided. This is just a single sample problem and similar other problems from socio-scientific domain would be developed to be used in the learning environment.

Research questions

The study will attempt to answer these four primary research questions.

1. To what extent does training in critical thinking skills help students identify fake news more effectively?
2. To what extent does collaborative learning help improve critical thinking skills among students while processing complex online information?
3. What are the various categories of learners in terms of their behavior while processing complex online information?
4. How do these various categories of learners differ in terms of use of critical thinking skills?

Method

Methodology

This research study will use a mixed-method approach with a heavy tilt towards the qualitative side. To answer the first two research questions, qualitative data would be required and this is the primary focus of this study. In order to understand the learner behavior and answer the last two research questions, a quantitative approach will be used which will employ learning analytics technique.

Target population and sampling

Even though digital literacy is a skill that is helpful at all ages of life, this study will be conducted primarily amongst undergraduate and postgraduate program students as individuals at this level are young adults and misinformation can lead them to take faulty steps at this critical juncture of life. Based on this misinformation, they might develop faulty beliefs which might get stay with them throughout their life.

Data Collection Tools

For the purpose of this study, multiple data sources will be utilized. While the students work in groups, the audio and video of their discussions will be recorded, along with screen recordings of their system and any notes they have created during the discussion. This will be followed by a follow-up interview to further probe their epistemic strategies.

The log data of the students while they interact with the system will also be collected. As explained in the proposed solution section, the log data from the system will contain time stamps which correspond to specific user activities in the system. This log data can be exported in the form of an excel data sheet which can be further processed to find user behavior patterns.

Finally, in order to measure the effectiveness of the learning environment on the critical thinking abilities of the students, a pre and posttest will be conducted. This test will present learners with various scenarios involving fake news.

Data Analysis

A rigorous and systematic methodology in the form of inductive coding strategy will be implemented in order to systematically examine and interpret the qualitative data that has been meticulously gathered through the various data collection tools. This approach will involve the identification of meaningful patterns, themes, and categories within the data, in order to uncover underlying meaning and to gain a comprehensive understanding of effectiveness of critical thinking in identifying fake news. The utilization of an inductive coding strategy will allow for the development of an inductive theory that is grounded in the data, and which can offer insight into the complex phenomena of critical thinking and fake news identification.

The log data, comprising of time-stamped information, will be subjected to a process of sequential pattern mining. This process will enable the identification of recurring patterns and sequences within the data, and the generation of clusters of users who exhibit similar patterns of behavior within the learning environment. By utilizing this approach, it will be possible to gain a deeper understanding of the usage patterns and behaviors of the users within the learning environment, which in turn can be used to understand the relation of certain behavioral patterns to demonstration of various critical thinking levels. Thus, the results generated by the sequential pattern mining algorithm will be used to model the behavior of users in terms of their display of critical thinking skills. This user behavior can be compared to the patterns displayed by professional fact checkers. In this way, desirable patterns of use of critical thinking skills can be identified which can be used for improving the critical thinking training aspects of the learning environment. This data can also be used to test new users and predict what kind of learning interventions would be required to help them develop necessary critical thinking abilities.

Dissertation status and next steps

The current research is a continuation of the work being done at the author’s organization in the field of technology enhanced learning of thinking skills. The organization in the past has conducted numerous studies ranging on various thinking skills from historical thinking, design thinking to estimation and more. There are also ongoing research projects that explore certain thinking skills like systems thinking. This current thesis research is a continuation of this work in terms of addressing more social problems like fake news and misinformation. This work draws from and builds on previous work done in the form of question posing and hypothesis testing skills amongst students.

The current author is currently in the early years of their PhD research and so the research plan is in in its nascent stage. Thus, a major reason for writing this paper is also to get helpful guidance from the members of the research community. Currently, the author has two immediate tasks in front of them. First, they plan to conduct a thorough literature review of critical thinking as a digital literacy skill, particularly in the educational context. The second task is to conduct a preliminary research study to explore how social interaction affects students' epistemic thinking in online inquiry learning. The author plans to use the following conjecture for this preliminary study: when working in groups, students' personal epistemologies will interact with each other and help to reflect on each other's cognitive biases and epistemic emotions.

Expected contributions

The present study is situated in a broad context of combating online fake news and equipping citizens with skills to process information overload. However, the novel contribution of this study will be to explore the effects of social interaction, in terms of collaborative work, on critical thinking abilities of students. This line of argument is rooted in the concept of democracy where vigilant citizens hold each other accountable in terms of their beliefs and practices. The research also has more immediate contributions in terms of understanding how different groups of people employ critical thinking while processing complex information online. This can help to create better resources for supporting people in spotting misinformation and debunking fake news. This whole process will affect society on two levels, at individual levels, people can become more conscious of their own biases and logical errors and a at a societal level, people can have more fruitful conversations across different thought camps as they will have a more solid ground of information to engage in discussion.

Aspects of the research on which advice is sought

Since the author is in their preliminary stage of research, they would be open to suggestions on almost every aspect of the study. However, the author is particularly interested in discovering more effective ways to capture critical thinking behavior of participants. As of now, the author is using a mixed-method approach and is relying on log data in terms of frequency and duration for capturing interaction. The author would like to receive more suggestions on how this aspect can be made more effective.

ACKNOWLEDGMENTS

The author expresses their sincere gratitude to Prof. Sahana Murthy and Prof. Ramkumar Rajendran from IIT Bombay for their invaluable guidance and support.

REFERENCES

1. Agesilaou, A., & Kyza, E. A. (2021). Empowering Students to be Data Literate: The Design and Implementation of a Learning Environment to Foster Critical Data Literacy. In Proceedings of the 15th International Conference of the Learning Sciences-ICLS 2021. (pp. 458-465). International Society of the Learning Sciences.
2. Alt News. (2021, June 11). About. Retrieved September 19, 2022, from https://www.altnews.in/about/
3. Association of College and Research Libraries. 2017. “Framework for Information Literacy for Higher Education.” Accessed March 4, 2017. http://acrl.ala.org/framework/
4. Barzilai, S., & Zohar, A. (2012). Epistemic thinking in action: Evaluating and integrating online sources. Cognition and Instruction, 30(1), 39-85.
5. Co-inform. (n.d.). The Project. Retrieved September 19, 2022, from https://coinform.eu/about/the-project/
6. Dede, C. (2008). A seismic shift in epistemology. Educause Review, 43, 80–81
7. Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking skills and Creativity, 12, 43-52.
8. Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational researcher, 18(3), 4-10.
9. Eshet, Y. (2004). Digital literacy: A conceptual framework for survival skills in the digital era. Journal of Educational Multimedia and Hypermedia, 13(1), 93–106, https://www.learntechlib.org/primary/p/4793/
10. Evans, J. S. B. (2003). In two minds: dual-process ac-counts of reasoning. Trends in cognitive sciences, 7(10), 454-459.
11. Evans, J. S. B., & Curtis-Holmes, J. (2005). Rapid responding increases belief bias: Evidence for the dual-process theory of reasoning. Thinking & Reasoning, 11(4), 382-389.
12. Evans, J. S. B., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on psychological science, 8(3), 223-241.
13. Ferrari, A. (2012). Digital competence in practice: An analysis of frameworks. JCR IPTS, Sevilla. https://ifap.ru/library/book522.pdf
14. Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn?. Educational psychology re-view, 16, 235-266
15. Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hämäläinen, R., Häkkinen, P., & Fischer, F. (2007). Speci-fying computer-supported collaboration scripts. Interna-tional Journal of Computer-Supported Collaborative Learn-ing, 2, 211-224
16. Kuhn, D., & Park, S. H. (2005). Epistemological understanding and the development of intellectual values. International Journal of educational research, 43(3), 111-124.
17. McGrew, S., & Glass, A. C. (2021). Click Restraint: Teaching Students to Analyze Search Results. In Proceedings of the 14th International Conference on Computer-Supported Collaborative Learning-CSCL 2021. (pp. 145-148). International Society of the Learning Sciences.
18. Ng, W. (2012). Can we teach digital natives digital literacy? Computers & Education, 59(3), 1065–1078. https://doi.org/10.1016/j.compedu.2012.04.016
19. O’Keefe, D. J. (Ed.) (2016). Social Judgment Theory. In Persua-sion: Theory and research (III, pp. 48–70). New Delhi: Sage Publications
20. Snopes. (2021, August 3). About Us. Retrieved September 19, 2022, from https://www.snopes.com/about/
21. Sulzer, A. (2018). (Re)conceptualizing digital literacies before and after the election of Trump. English Teaching: Practice & Critique, 17(2), 58–71. https://doi.org/10.1108/ETPC-06-2017-009

© 2023 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.