ABSTRACT

We report on a study in which we examined the work habits of six students who agreed to use do not disturb on their phone while working on programming assignments. Two students tried do not disturb, and quickly quit using it. Three out of four remaining student participants were more productive while using do not disturb when working on their projects. These students exhibited different work patterns, different levels of receptiveness, different use of do not disturb, and different phone use, suggesting that using do not disturb has the potential to increase productivity for some students. Our study provides the first quantitative measurements of productivity with phone/no phone conditions in a computing education setting by using keystroke data. In addition, our study is the first that we know of in any setting that measures engagement with high temporal fidelity. Additionally, we surveyed 195 computer science students about their use of do not disturb and willingness to try it. About half of respondents said that they have used do not disturb and most of them use it at least occasionally.

Keywords

1. INTRODUCTION

Two recent studies have shown that Introductory Computer Programming (CS1) students disengage approximately 20 times on average while working on computer programming assignments [9, 17]. Most of these breaks are short, usually 1 to 4 minutes. While the studies made no claims as to what students were doing during those breaks, the authors conjectured that cell phone use could be a major factor. Indeed, 1 to 4 minutes is just enough time to receive a phone notification or message a friend. Luckily for students, most phones come with a Do Not Disturb (DND) setting that will block most notifications when it is turned on, a perfect solution to those disruptive notifications; however, many users do not utilize DND as a tool to improve focus because cell phone users view notifications as their indispensable connection to information, social environments, and environmental surveillance [26].

It would seem obvious that turning on DND would decrease distractions and interruptions and increase task engagement, and studies have shown that people feel more productive when DND is on [31]; however, studies have also shown that people, especially those who exhibit need to belong NtB¹ or fear of missing out FOMO², think about their phones more when DND is on and distract themselves by checking their phones, unprompted, more than they would have with DND off³ [12, 26].

In thispaper, we present a case study in which six CS1 students agreed to use DND on their phones during half of their programming assignments. We then measure productivity by logging keystrokes and comparing the frequency and length of breaks between the two DND off/on conditions. Our work seeks to discover if using DND benefits students when used during their programming assignments by answering the following research questions:

• How does using the DND setting on a phone while working on programming assignments affect task engagement and productivity?
• Are CS1 students receptive to using DND?

We find evidence that DND increases task engagement and productivity in some CS1 students by at least 5% to 12% (Section 4.1 and Section 4.1.2). We also find a reluctance to use DND during assignments, or even participate in research about DND (Section 4.2.3). Many students have tried DND already, but the cost to recruit students to DND research is non-trivial (Section 4.2.4).

Our data supports the notion that using DND during assignments can help some students be more productive as an immediate benefit.

2. RELATED WORK

2.1 Task Engagement

Leinonen et. al. [24] argue that time-on-task (TOT) is one of the most important metrics for learning and achievement because it correlates well with performance; an argument well defended in research [5, 2, 37, 19]. Chickering and Gamson even argue that “Time plus energy equals learning" [6]. TOT is not just elapsed time, it is time spent engaged — time plus energy. If students spend time and energy on unrelated tasks while putatively working on their assignments, it may lead them to feel they have worked harder for longer on an assignment than they actually have.

Computer Science is an appropriate field in which to study time on task and engagement because keystroke data allows researchers to capture the step-by-step process students use to write code [8]. Recent work has shown that keystroke data can be used to accurately estimate time-on-task in computer science classes [9, 17]. Keystroke data provides such high-resolution data that the author of a document can be identified by their typing patterns [27], plagiarism can be better detected and deterred [18, 16], and differences in context can be discovered [10]. Keystroke data has the potential to discover fine-grained differences in behaviors when DND is on or off instead of relying on self-reported data (i.e. “I felt more productive") [31].

When working on important tasks, it is beneficial to not be distracted because working memory will be consumed by the distraction, and then time will be needed to restore the lost working memory, a concept known in cognitive psychology as task switching. The cost of a task switch can be measured by measuring performance with and without task switching [33]. Preferential focus is given to stimuli relevant to current working memory [29, 36], but stimuli relevant to long-term or self-relevant goals may distract from the current task  [35] (e.g., hearing your own name [28]). Cell phone integration into human life has caused an individual’s phone notification sounds to be self-relevant, causing an involuntary attention response similar to hearing one’s own name [34].

2.2 Phone Notifications

End et. al. conducted a study where participants were asked to watch an educational video, take notes, then take a test on the material presented in the video [11]. Participants were divided into two groups: ringing and nonringing. In the ringing group, a cell phone rang twice during the video, and participants in the ringing group scored significantly lower than the nonringing group.

Researchers often view phone notifications as distractions, especially in the context of driving [40, 4]. Notifications grab attention  [38], decrease productivity at work [20, 31], and reduce learning in class [15], even if the notifications are not given immediate attention [38, 13]. When notifications are heard and not given attention, or expected and not heard, thoughts drift away from the task at hand and towards the notification. In contrast to researchers’ views that notifications are distractions, cell phone users view notifications as their indispensable connection to incoming information, social environments, and environmental surveillance [26].

In the context of self-paced tasks, the performance cost of a phone notification could be mitigated to time only if the user could bring their attention back to their task by restoring relevant content to their working memory. Fox et. al. and Bowman et. al. showed that students who use instant messaging while working on assignments take longer to complete their assignments, but do not perform worse [14, 3].

In the context of non-self-paced work, notifications appear to be more damaging. Simply ignoring phone notifications may not be effective, phone notifications alone (without interacting with the phone) can cause task irrelevant thoughts and mind wandering which reduce task performance, even causing impulsive phone checking [38]. Even the “mere presence" of a phone can be distracting and reduce task performance in attention-demanding tasks [39]. Ward et. al. argue that the presence of a phone causes a “brain drain" reducing available cognitive capacity to engage in the task at hand [44].

2.3 Do Not Disturb

Do not disturb (DND) is a feature on phones that blocks non-emergency notifications. Pielot et. al. [31] conducted a 24-hour no-notification study and found that participants felt more productive, anxious, and lonely with DND on than with DND off. They found that some participants felt anxiety over missing notifications and not being responsive, leading them to check their phones more often. Others felt less stress from not having notifications. Participants in Pielot et. al.’s s study said they would continue to use DND and 59.1% of participants who said they would use DND in the future indicated that they did in a follow-up interview two years later. Exposure to do-not-disturb may encourage people to use it on their own.

It comes as no surprise that the number of phone notifications is positively correlated to the number of times a phone is picked up and how long it is in use [26]. It would make sense that DND would reduce the number of notifications, and therefore phone use, but using do-not-disturb to limit screen time, or limit phone use, in general, may be counterproductive, and increase screen time [26]. This phenomenon seems to happen because people are worried about missing important notifications, especially people with NtB and FOMO [46, 12, 26]. Some people even report phantom vibrations – feeling a phone vibrate when it did not – possibly because they are expecting notifications to come [7, 21, 22].

A surprising gap in current literature is the lack of research on using DND for a single task; we are only aware of Zamani et. al.’s work on using DND during discharge prescription writing [47]. Studies have focused on the effects of using DND for extended periods of time and have found that DND is not a good solution for long periods of time. Fitz. et. al. found that when notifications were batched and delivered a couple of times throughout the day, users had an increase in attentiveness, increase in productivity, boosted mood, and an increase in feelings of control [12]. Nevertheless, few users change their notification settings on their devices [43].

2.4 Phone Addiction

The utility of cell phones has caused them to become a constant companion in almost every aspect of human life. The United States Supreme Court said in a 2014 case, “[Cell phones] are now such a pervasive and insistent part of daily life that the proverbial visitor from Mars might conclude they were an important feature of human anatomy” [41].

3. METHODS

3.1 Phone Studies 1 and 2

Our first research question is: How does using the DND setting on a phone while working on programming assignments affect task engagement and productivity? We designed a study that correlates typing behavior, through keystroke logging, with the DND setting on participants’ phones. We used three pieces of software to do this (Figure 1). The first is the ShowYourWork (SYW) [8] plugin to the PyCharm IDE. SYW keeps a log of keystrokes which the student submits to our learning management system (LMS) together with their programming solution. Our CS1 courses require all students to use PyCharm with the ShowYourWork plugin. The second piece of software is the DND PyCharm plugin that pseudo-randomly chooses assignments for the student to use DND on and tells the student when to turn on DND. The third piece of software is a custom Andoid app called DND. When the user opens the DND app, do not disturb is turned on and a log is made to a server. If the user closes or navigates away from the DND app, do not disturb is turned off and a log is made to the server.

This research was conducted over three semesters of CS1 at a mid-sized U.S. university: study 1 in spring 2023 and study 2 in fall 2023. The studies were identical, as described below, with the following exceptions. Participants in study 1 were compensated $7 (USD) in compensation and participants in study 2 were compensated $30. Before study 2, participants in study 1 were given an additional $23 to make compensation equal. Another difference was that students in study 1 were compensated if they signed up for the study, but compensation for study 2 was conditional on using DND when instructed. A third difference is that in study 2, students were invited to allow researchers to use their keystroke logs even if they weren’t participating by using DND. These students were not compensated and served as a control group in our analysis of study 2 data. For both studies, participants installed the DND phone app and the DND PyCharm plugin. As noted above, the SYW plugin was already installed as it was required of all students in the course.

Nine students signed up to participate in study 1 but only eight installed the phone app, three installed the plugin, and three used the phone app during at least one assignment. We refer to these students as Student1, Student2, and Student3. In study 2, nine students signed up to participate, three students used the phone app, but only two used it every time it was requested. The three students that used the phone app at least once are referred to as Student4, Student5, and Student6. Also in study 2, 35 students allowed us to use their keystroke logs, enabling us to control for different assignment difficulties.

During the study, the PyCharm DND plugin randomly selected half of a participant’s assignments and show a pop-up window that requested participants to use the phone app while working on that assignment. The phone app enabled DND when it was opened, disabled DND when closed or put in the background, and logged DND status to a remote database. Keystroke data was available with assignment submissions due to the course requirement that students install the ShowYourWork PyCharm plugin that logs keystrokes. Participants were assigned a random 6-digit ID to protect their identities. This research was reviewed and approved by our university’s ethics review board (IRB:13030).

Participants filled out surveys at the beginning and at the end of the academic term during which the studies were carried out. The survey questions were designed to help answer research question 2 by discovering if students thought their phones distracted them, if they used DND already, and if they recognized any effects from using DND. The survey questions and answers are in Table 4.

3.2 Survey Study

Our second research question is: Are CS1 students receptive to using DND? To help answer this research question and, in addition, to determine if insufficient compensation was a contributing factor to our low participation, we sent a link to the survey shown in Figure 2 to our computer science graduate and undergraduate email list.

3.3 Statistical Tests

We report \(p\) values of all statistical significance tests, of which there are four. We follow the American Statistical Association’s recommendations to use \(p\) values as one piece of evidence of significance, to be used in context [45].

4. RESULTS

4.1 Effects on Engagement and Productivity

Our first research question is: How does using the DND setting on a phone while working on programming assignments affect task engagement and productivity? We answer the question by investigating the difference in keystroke latencies (elapsed time between keystrokes) with DND in the on and off conditions. We expect that a distraction from a phone would cause a long latency keystroke to happen because the student would stop working to respond to the notification or be distracted even if they didn’t respond. We expect that using DND would remove some distractions and therefore some long latency keystrokes.

4.1.1 Phone Study 1.

We performed a chi-squared test on Student1’s and Student2’s typing behaviors. Student3 tried DND briefly, then never used it again; making too few keystrokes with DND on to perform a test using their data. See Table 1. The variables are bin (0-15 seconds, 15-60 seconds, 1-4 minutes, and 4-60 minutes) and DND status (ON, OFF). Values are total keystroke counts. DND is found to affect typing behavior for Student1, \(\chi ^2(3, N=113,446)=12.2, p=0.0068\), and also for Student2, \(\chi ^2(3, N=18,222)=9.09, p=0.028\). Figure 3 shows that when DND is on, students make fewer long latency keystrokes suggesting an increase in task engagement and productivity.

We can estimate a minimum time saved by removing a percentage of long latency keystrokes from assignments where DND was not used. For example, Student1 spent 15 hours working on six assignments without DND. With the reduction of 16% of keystrokes with latencies of .25 to 1 minute when using DND, Student1 could have saved at least 17.5 (2%) minutes. Applying the same function to the 1-4 minute bin yielded an additional 35 (4%) minutes of time saved for a total of 6% time savings at minimum. Similarly, Student2 could have saved at least 8% of their time. These time savings did not go unnoticed by Student2 who indicated in the post-survey that they felt more productive with DND on. We can estimate that students saved at least 6% to 8% of their time by using DND in Study 1.

Remarkably, Student1 and Student2 showed near opposite willingness to use DND while working on assignments – Student1 used DND even when not asked and Student2 did not always use DND when asked – but they both demonstrated a similar reduction in long latency keystrokes when DND was used. While students may have different opinions or preferences about using DND, it is clear that there may be a time-saving benefit when using DND. In the case of Student3, we do not know if using DND would have been beneficial. It is possible that DND could have helped Student3 if they tried it longer and became accustomed to using it; however, Student3 reported that using DND demotivated them to work on their assignments and quit participating in our study. See Section 4.2 for more details on Student3.

4.1.2 Phone Study 2.

This study was designed with a control group in order to control for differences in pausing behaviors across assignments [42]. In this study we collected keystroke data from 38 students, 3 of whom used DND: Student4, Student5, and Student6. Using this data we can examine the effects of using DND relative to other students.

In this analysis, our operational definition of “task engagement and productivity" (terms used in our first research question) is the number of 5+ minute breaks students take per 1000 keystrokes. We call this the break rate. We interpret higher break rates to mean the student is less engaged. Since typing and pausing behavior may change from assignment to assignment for a given student [42], we analyze participants’ break rates relative to other students for each assignment. Tables 2 and 3 show Student4 and Student5’s break rate rank among other students for each file, normalized to 100 students. For example, if the student’s rank is 25, then 75% of the other students had a higher break rate. A lower rank can be interpreted to mean that the student was relatively more engaged than most other students.

Figure 3a shows that when Student4 used DND they tended to be more engaged relative to other students. A Mann-Whitney U test gives evidence that the difference in engagement (median of 19.5 when in DND mode and 52.0 when DND is off) is indeed linked to the DND status (\(U=1.5\), \(p=0.049\), two-tailed). However, this effect may not be present for all students, as the difference in medians for Student5 (median of 50.0 in DND mode and 53.0 when DND is off) is not statistically significant (\(U=12.0\), \(p=0.67\), two-tailed). It is clear that for one student DND makes a big difference, but DND may not make as big of a difference for the other student.

We can compare a students typing behaviors to the course average to control for differences in assignments. Figure  5 shows that Student4 completed assignments faster than the class on average when not using DND and completed assignments even faster with DND on. Without DND, Student4 completes assignments 17% faster than the class on average, and with DND Student4 completes assignments 27% faster (a 12% increase in productivity). Student5 also sees an increase in productivity, although much smaller at 5% (see Figure  6). Student4 and Student5 also exhibit an increase in average latency in keystrokes with latency over 15 seconds. This suggests that DND stops some notifications that would distract students for a short time, reducing some short duration disengagements. Results from Study 2 help to validate our findings in Study 1 and provide additional evidence that DND can increase student productivity between 5% and 12%.

4.2 Receptivity to Using DND

Our second research question is: Are CS1 students receptive to using DND?

4.2.1 Phone Study 1.

In phone study 1, three participants used DND while working on their assignments: Student1, Student2, and Student3. Each of the three participants has shown remarkably different interactions with DND while working on assignments. The most engaged participant was Student1. At the start of the semester, Student1 used the app when requested and did not use the app when not requested. Then after using the DND for a few assignments, started using DND for almost every assignment whether asked to or not. Student1 completed most assignments within a single session, working to completion without engaging in other activities (Figure 6a).

Student2 was less receptive to DND. For the first assignment, Student2 used DND during the entire duration of their assignment. In the following assignments when asked to use DND, Student2 only used DND 7.2% of the time, then 18% of the time, then not at all. Student2 worked on assignments in many sessions and did not submit a solution to 4 of the 10 assignments.

Student3 was the least receptive. When asked to use DND for the first time, Student3 briefly used DND but then turned it off. In an email to a researcher, the student explained that they began to avoid doing the assignment because they felt like they could not focus without using their phone for other things. Student3 realized that using DND was harmful to their motivation and removed the DND app from their phone. Student3 worked on assignments over many sessions with the longest amount of time between the first and last keystroke in an assignment. See Figure 6c. Due to the lack of data for Student3, limited analysis could be done.

In the pre-survey responses, we found that Student2 and Student3 felt that their phone distracted them while doing homework, spending 60% of their time on their phones. Student1 did not feel distracted by their phone, only spending 10% of their time on their phone (Table 4). In the post-survey responses, Student2 reported being more distracted by their phone than Student1, and liked using DND more than Student1. In addition, Student2 felt more efficient with DND and more likely to use DND in the future. We did not find that our participants thought about their phones more often when DND was on in our survey, but Student3 quit participating because they did not want to use DND, according to the email communication. It is likely that Student3 thought about their phone more often.

4.2.2 Phone Study 2.

Student4 used DND during every assignment whether it was requested or not until we sent an email letting them know their participation was considered complete, at which point they did not use DND again. Student5 was only requested to use DND during a single assignment – due to the selection being random – and used DND as requested but never used DND when not requested. Student6 used DND briefly during one assignment then did not use DND again. It is likely that Student6 was not receptive to DND and decided the money wasn’t worth it. Due to the lack of data for Student6, limited analysis could be done on their data.

4.2.3 Studies 1 and 2 Surveys.

Participants were more efficient while using DND but seemed unwilling, or uncommitted, to using DND in the future. As discussed in Section 3, many students value receiving notifications while doing assignments more than the $30 compensation offered to students for participation in the study. This may depend on sociability; more sociable students are likely to use their phones more [25], increasing both the cost and potential benefits of using DND. Our method of using DND for a single task seeks to be a minimal change to a student’s environment and reduce the potential anxiety that students may feel while using DND. Using DND for a single task allows students to receive the benefits of DND without the cost of losing their phone notifications for too long.

Our participants reacted very differently to DND, and their survey responses are counter-intuitive in some cases. Student1 used DND even when not asked but said they didn’t like or dislike using DND and did not plan on using DND in the future. Student1 may have used DND even when not asked because they felt more productive or liked using DND, but their survey responses show that they were neutral to DND. It is possible that Student1 used DND and felt more productive at first, but then as they continued to use it lost the feeling of increased productivity and quit using DND all the time (Figure 6a, Table 4).

Student2 rarely used DND when asked, but said they liked using it but were unsure if they would use it in the future. We do not know why they did not use it more often if they liked using it or why they were unsure about using it in the future. We think this could be an example of phone addiction; the student knew they were distracted by their phone, and knew using DND helped them stay focused, and increase productivity, but is unwilling, or unable, to use DND when asked or in the future.

Student3 quit using DND shortly after they tried it. According to an email communication, Student3 felt that their phone is an indispensable tool to “regroup [their] thoughts", and losing that tool (by using the phone app) caused them to avoid doing their assignments. Student3 is an example of someone who may be harmed by requiring DND, and evidence that the use of DND should be voluntary.

4.2.4 Survey Study.

We sent a survey to our university’s graduate and undergraduate students. We then asked CS1 instructors to post our survey to their courses Canvas pages to increase participation from CS1 students. We collected 128 completed responses, including 30 from current CS1 students. We found that about half of our respondents have tried DND (see Figure  8) and of those that have tried DND most of them use DND at least occasionally (Figure  9).

We also find that our compensation should have been adequate for student participation (Figure  10). Our $30 compensation should have been enough for 25 students to participate in our study yet only 3 did. To verify that students understood our question about the cost to participate, we explicitly stated “how much money would it take to get you to participate in a study" and included a second question that asked about the cost to use DND without study participation. We find that the cost to participate in DND research is not a simple question of compensation because our compensation was large enough to have at least an eight times higher participation than we observed. Students may think some level of compensation is high enough but when they are asked to actually sign up, or begin participation, decide it is not enough, as we possibly observed with Student6. It is possible that higher compensation would attract more students, but determining the cost will be difficult.

5. THREATS TO VALIDITY

Our study has a low number of participants, but many samples (keystrokes) from each participant. So the results are not strongly generalizable, but the evidence of the effects of DND for individual participants is strong. A threat to internal validity is that students may only use DND when they want to focus or know they have significant time to commit to their assignment leading to a sampling error. We do not think this affected our findings from phone study 1 because Student1 completed their assignment in single sessions and had similar results to Student2 who could have exhibited this behavior. Nor do we think it affected phone study 2 as Student4 and Student5 largely used DND when asked.

6. CONCLUSIONS

Three out of our four engaged study participants were more productive, in terms of fewer pauses and shorter development times, when using DND on their phones. We also find that, while many students report positive attitudes toward using DND, many students are nevertheless reluctant to actually use it. We had difficulty recruiting and retaining students in our study even when $30 in compensation was offered for participation.

Our studies contribute additional evidence to the body of knowledge that phone notifications cause distraction and disengagement and that removing phone distractions by using the do not disturb phone feature can improve productivity. Beyond additional evidence, our work contributes unique insights by using higher temporal resolution measurements (keystrokes) than previous work. In this way, we have been able to make disengagement and productivity claims at the scale of minutes, giving a more complete picture of how the phone can affect productivity on even short tasks. We also contribute one of the first ways to quantitatively measure engagement in a natural, non-invasive, non-laboratory setting. Our methodology of using keystrokes to measure engagement is an important contribution that we expect will lead to further understanding of phone usage and its relation to productivity.

7. REFERENCES

1. R. F. Baumeister and M. R. Leary. The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychological bulletin, 117(3):497, 1995.
2. B. S. Bloom. Time and learning. American psychologist, 29(9):682, 1974.
3. L. L. Bowman, L. E. Levine, B. M. Waite, and M. Gendron. Can students really multitask? an experimental study of instant messaging while reading. Computers & Education, 54(4):927–931, 2010.
4. J. K. Caird, K. A. Johnston, C. R. Willness, M. Asbridge, and P. Steel. A meta-analysis of the effects of texting on driving. Accident Analysis & Prevention, 71:311–318, 2014.
5. J. B. Carroll. A model of school learning. Teachers college record, 64(8):1–9, 1963.
6. A. W. Chickering and Z. F. Gamson. Seven principles for good practice in undergraduate education. Biochemical Education, 17(3):140–141, 1989.
7. A. Deb. Phantom vibration and phantom ringing among mobile phone users: A systematic review of literature. Asia-Pacific Psychiatry, 7(3):231–239, 2015.
8. J. Edwards, K. Hart, R. Shrestha, et al. Review of csedm data and introduction of two public cs1 keystroke datasets. Journal of Educational Data Mining, 15(1):1–31, 2023.
9. J. Edwards, K. Hart, and C. Warren. A practical model of student engagement while programming. In Proceedings of the 2022 ACM SIGCSE technical symposium on computer science education, 2022.
10. J. Edwards, J. Leinonen, and A. Hellas. A study of keystroke data in two contexts: Written language and programming language influence predictability of learning outcomes. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education, pages 413–419, 2020.
11. C. M. End, S. Worthman, M. B. Mathews, and K. Wetterau. Costly cell phones: The impact of cell phone rings on academic performance. Teaching of Psychology, 37(1):55–57, 2009.
12. N. Fitz, K. Kushlev, R. Jagannathan, T. Lewis, D. Paliwal, and D. Ariely. Batching smartphone notifications can improve well-being. Computers in Human Behavior, 101:84–94, 2019.
13. P. E. Fortin and J. Cooperstock. Understanding smartphone notifications’ activity disruption via in situ wrist motion monitoring. In CHI Conference on Human Factors in Computing Systems Extended Abstracts, pages 1–6, 2022.
14. A. B. Fox, J. Rosen, and M. Crawford. Distractions, distractions: does instant messaging affect college students’ performance on a concurrent reading comprehension task? CyberPsychology & Behavior, 12(1):51–53, 2009.
15. K. Graben, B. K. Doering, and A. Barke. Receiving push-notifications from smartphone games reduces students learning performance in a brief lecture: An experimental study. Computers in Human Behavior Reports, 5:100170, 2022.
16. K. Hart, C. Mano, and J. Edwards. Plagiarism deterrence in cs1 through keystroke data. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, pages 493–499, 2023.
17. K. Hart, C. Warren, and J. Edwards. Accurate estimation of time-on-task while programming. In Proceedings of the 2023 ACM SIGCSE technical symposium on computer science education, 2023.
18. A. Hellas, J. Leinonen, and P. Ihantola. Plagiarism in take-home exams: help-seeking, collaboration, and systematic cheating. In Proceedings of the 2017 ACM conference on innovation and technology in computer science education, pages 238–243, 2017.
19. N. Karweit. Time-on-task reconsidered: Synthesis of research on time and learning. Educational leadership, 41(8):32–35, 1984.
20. Y.-H. Kim, J. H. Jeon, E. K. Choe, B. Lee, K. Kim, and J. Seo. Timeaware: Leveraging framing effects to enhance personal productivity. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 272–283, 2016.
21. D. J. Kruger and J. M. Djerf. High ringxiety: Attachment anxiety predicts experiences of phantom cell phone ringing. Cyberpsychology, behavior, and social networking, 19(1):56–59, 2016.
22. D. J. Kruger and J. M. Djerf. Bad vibrations? cell phone dependency predicts phantom communication experiences. Computers in Human Behavior, 70:360–364, 2017.
23. M. R. Leary, K. M. Kelly, C. A. Cottrell, and L. S. Schreindorfer. Construct validity of the need to belong scale: Mapping the nomological network. Journal of personality assessment, 95(6):610–624, 2013.
24. J. Leinonen, F. E. V. Castro, and A. Hellas. Time-on-task metrics for predicting performance. In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education, pages 871–877, 2022.
25. L. Leung and R. Wei. More than just talk on the move: Uses and gratifications of the cellular phone. Journalism & mass communication quarterly, 77(2):308–320, 2000.
26. M. Liao and S. S. Sundar. Sound of silence: Does muting notifications reduce phone use? Computers in Human Behavior, 134:107338, 2022.
27. K. Longi, J. Leinonen, H. Nygren, J. Salmi, A. Klami, and A. Vihavainen. Identification of programmers from typing patterns. In Proceedings of the 15th Koli Calling conference on computing education research, pages 60–67, 2015.
28. N. Moray. Attention in dichotic listening: Affective cues and the influence of instructions. Quarterly journal of experimental psychology, 11(1):56–60, 1959.
29. G. B. Moskowitz. Preconscious effects of temporary goals on attention. Journal of Experimental Social Psychology, 38(4):397–404, 2002.
30. U. Oberst, E. Wegmann, B. Stodt, M. Brand, and A. Chamarro. Negative consequences from heavy social networking in adolescents: The mediating role of fear of missing out. Journal of adolescence, 55:51–60, 2017.
31. M. Pielot and L. Rello. Productive, anxious, lonely: 24 hours without push notifications. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, pages 1–11, 2017.
32. A. K. Przybylski, K. Murayama, C. R. DeHaan, and V. Gladwell. Motivational, emotional, and behavioral correlates of fear of missing out. Computers in human behavior, 29(4):1841–1848, 2013.
33. R. D. Rogers and S. Monsell. Costs of a predictible switch between simple cognitive tasks. Journal of experimental psychology: General, 124(2):207, 1995.
34. A. Roye, T. Jacobsen, and E. Schröger. Personal significance is encoded automatically by the human brain: an event-related potential study with ringtones. European Journal of Neuroscience, 26(3):784–790, 2007.
35. R. M. Shiffrin and W. Schneider. Controlled and automatic human information processing: Ii. perceptual learning, automatic attending and a general theory. Psychological review, 84(2):127, 1977.
36. D. Soto, D. Heinke, G. W. Humphreys, and M. J. Blanco. Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology: Human Perception and Performance, 31(2):248, 2005.
37. J. Stallings. Allocated academic learning time revisited, or beyond time on task. Educational researcher, 9(11):11–16, 1980.
38. C. Stothart, A. Mitchum, and C. Yehnert. The attentional cost of receiving a cell phone notification. Journal of experimental psychology: human perception and performance, 41(4):893, 2015.
39. B. Thornton, A. Faires, M. Robbins, and E. Rollins. The mere presence of a cell phone may be distracting. Social Psychology, 2014.
40. J. E. Törnros and A. K. Bolling. Mobile phone use—effects of handheld and handsfree phones on driving performance. Accident Analysis & Prevention, 37(5):902–909, 2005.
41. Riley v. california, 2014.
42. J. Urry and J. Edwards. A Framework that Explores the Cognitive Load of CS1 Assignments Using Pausing Behavior. In ACM Technical Symposium on Computing Science Education (SIGCSE), 2024.
43. A. Voit, D. Weber, and N. Henze. Qualitative investigation of multi-device notifications. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, pages 1263–1270, 2018.
44. A. F. Ward, K. Duke, A. Gneezy, and M. W. Bos. Brain drain: The mere presence of one’s own smartphone reduces available cognitive capacity. Journal of the Association for Consumer Research, 2(2):140–154, 2017.
45. R. L. Wasserstein and N. A. Lazar. The asa statement on p-values: context, process, and purpose, 2016.
46. S. Yoon and K.-p. Lee. A study on notification system design of smartphone messenger considering the user’s stress. Archives of design research, 28(2):75–88, 2015.
47. M. Zamani, K. Hall, A. Cunningham, N. Chin, S. Kent-Ferguson, and V. Wadhwa. Effectiveness of ‘do not disturb’strategies in reducing errors during discharge prescription writing. Journal of Pharmacy Practice and Research, 49(5):433–438, 2019.