ABSTRACT

Within the past 10 years there has been increasing momentum of the open science movement to make research more open, transparent, and reproducible. However, the adoption of open science practices in education lags behind other fields. In this hybrid tutorial, we will begin by providing a brief overview of open science practices, benefits and workarounds, as well as how the statistical foundations of open science, including the benefits for inference and hypothesis testing. In the second part of the workshop, we will provide a hands-on tutorial of how to use the Open Science Framework to make projects, invite collaborators, preregister studies, share data, code, and materials. Participants in this workshop will gain a better understanding of open science practices, the reasons motivating their adoption, and how to use the Open Science Framework to make their research more open.

Keywords

open science, open access, preprints, preregistration

Background

The adoption of open science practices– such as preregistration, open data, open code and materials, preprints, and open access– has become increasingly normative across various scientific disciplines, yet attitudes and adoption of these practices remain lagging in education science [1]. While some sub disciplines of education have begun to pioneer the open science movement in education (e.g. special education research, [2]; gifted education, [3]) remaining subfields, such as those related to educational data mining, have not seen widespread adoption. There are many reasons that education scientists may not have adopted open science practices, whether from a lack of training, to general concerns about what can and cannot be shared [4]. Despite the reason, there is a greater need for education about what open science is, how it can be leveraged, as well as guidance on how to use existing resources to make education research more open and transparent.

In this tutorial, we will cover the basics of open science practices, discuss the specific challenges associated with education research (e.g. protecting participant anonymity), and provide a step-by-step tutorial of how to use the Open Science Framework to preregister a study, post open data, code, and materials, and post preprints. We will also discuss further open science activities such as registered reports that might be relevant for participants, and how to use other tools to identify opportunities to publish open access (e.g. directory of open access journals, sherpa romeo, etc.).

Format and Timeline of Tutorial

The proposed tutorial will be offered in a hybrid fashion, and focuses on open science in education research, and using the Open Science Framework to preregister studies, share data and code, and post preprints. An outline of this workshop can be found below.

• First, we will provide a brief overview of some of the problems of conducting science and research, how open science practices are being used to overcome some of these issues, as well as the myths and workarounds of these practices.
• Second, we will discuss the statistical foundations of open science, including the benefits for inference and hypothesis testing. This discussion will include information about which aspects of data analysis may themselves depend on the data–and hence do not need to be pre-registered, and which phases may not.
• For the third part of this tutorial, we will lead a hands-on tutorial on how to navigate the OSF website, make an account, create projects, invite contributors to join projects, and how to post projects for the public. Afterwards, the presenter will give a step-by-step guide on how to create a preregistration, discuss best steps for preregistration, and identify how to create an embargo on a given preregistration. Other features, such as how to create anonymous versions of projects for blind peer-review will be shown.

REFERENCES

1. Nosek, B. (2022). Making the Most of the Unconference. Presented at the Unconference on Open Scholarship Practices in Education Research. Available at https://osf.io/9k6pd/
2. Makel, M. C., Smith, K. N., McBee, M. T., Peters, S. J., & Miller, E. M. (2019). A path to greater credibility: Large-scale collaborative education research. AERA Open, 5(4). https://doi.org/ 10.1177/2332858419891963
3. Cook, B. G., Lloyd, J. W., Mellor, D., Nosek, B. A., & Therrien, W. J. (2018). Promoting open science to increase the trustworthiness of evidence in special education. Exceptional Children, 85(1), 104– 118. https://doi.org/10.1177/0741932516637198
4. Logan, J. A., Hart, S. A., & Schatschneider, C. (2021). Data sharing in education science. AERA Open, 7, 23328584211

© 2022 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.