ABSTRACT

360-degree classroom video captures entire learning spaces using omnidirectional recording, preserving group-level context and reducing blind spots associated with fixed-view cameras. This affords advantages for analyzing non-verbal behaviors central to understanding collaborative interaction. However, manual annotation of such video remains time-consuming and difficult to scale. Multimodal large language models (MLLMs) offer a promising alternative: unlike traditional computer vision models that often struggle with the curvilinear distortion and viewpoint variation, MLLMs can leverage global context and semantic cues to interpret complex scenes. Yet applying MLLMs to 360-degree video requires decisions about how panoramic scenes are represented and how temporal information is sampled, factors known to influence model performance but rarely examined in educational contexts. This exploratory case study investigates how video extracted views (front-facing vs. top-down view) and temporal sampling density (1–4 frames per second) influence MLLM-based detection of non-verbal behaviors. We applied Molmo2-8B to 360-degree classroom recordings of small groups collaborative learning and evaluated detection performance across gaze, gesture, posture and tool usage behaviors. Results reveal that object-related behaviors (e.g., gaze at laptop, tool usage) were detected more reliably than social behaviors requiring inference about interpersonal coordination (e.g., gaze at other students). Projected view shaped detectability: top-down views supported more reliable detection of gaze- and tool-related behaviors by preserving spatial context. However, higher sampling rates did not consistently improve detection, suggesting that more frames are not always better. Together, these findings can inform 360-degree video preprocessing decisions when applying MLLMs to video analysis.

Keywords

1. INTRODUCTION

Understanding collaborative learning requires attention not only to what learners say or produce, but also to how they coordinate through non-verbal behavior [14]. Gaze, gesture, posture, and shared tool use play important roles in establishing joint attention, signaling understanding, and regulating turn-taking during group work [3, 20, 16]. Research in learning sciences has long recognized these behaviors as indicators of productive collaboration—moments when learners build shared understanding, negotiate meaning, or struggle to coordinate [40, 43]. Capturing these dynamics is essential for understanding how collaboration unfolds and for designing interventions that support effective group learning [15, 50]. Yet, non-verbal behaviors remain difficult to capture through interaction logs or textual data alone, making classroom video a central data source for examining collaborative learning processes [13, 25].

To capture these non-verbal dynamics in situ, researchers have relied on classroom video recordings. Traditional front-facing cameras require researchers to make early decisions about camera placement and viewing angle, which can result in occlusion or loss of information when learners move outside the camera’s limited field of view [10]. These constraints are especially problematic in collaborative learning settings. Recent advances in video capture technology have sought to address these limitations. In particular, wide angle and 360-degree cameras enable capture of entire learning spaces, reducing blind spots and preserving group-level context across multiple learners [2, 39, 47]. Although 360-degree recording addresses some of these limitations, it introduces new challenges. Researchers need to reconstruct and reprocess video from distorted equirectangular projection, and manage the increased complexity of processing panoramic footage. These technical demands compound the already substantial cost of manual video annotation, which has long constrained the scale of research on collaborative learning, often limiting analysis to small samples or brief episodes [58]. Prior work has explored computer vision techniques to automate classroom video analysis, including methods for detecting gaze direction, gesture, and body posture [21, 62, 57]. These approaches have shown promise but typically rely on task-specific pipelines and can be sensitive to viewpoint variation and visual distortion in 360-degree video [6, 19].

Multimodal large language models (MLLMs), a class of vision-language models capable of processing video input, have recently emerged as a promising alternative [23, 59]. By integrating visual perception with natural language generation, MLLMs offer a unified interface for describing a wide range of behaviors, potentially lowering the technical barrier to video-based research [51]. However, existing research provides limited guidance for educational applications. Empirical evaluations of MLLMs primarily rely on standardized benchmarks developed outside educational domains [53, 56], offering limited insight into how MLLMs perform on the context-dependent behaviors that characterize learner interactions. Meanwhile, work on MLLMs in education has remained largely conceptual rather than empirically examining model performance in authentic learning settings [5]. There is also limited understanding of how analytic conditions shape MLLM performance, as model outputs are impacted not only by architecture but also by representation choices and dataset characteristics [11]. These factors are particularly consequential for 360-degree video, where researchers must make explicit decisions about how panoramic footage is projected, framed, and sampled before it can be processed by a model. Yet how such preprocessing decisions affect MLLM performance remains an open question.

This study addresses this question by empirically examining two analytic design choices: video extracted views (how the panoramic scene is projected and framed) and temporal sampling (how many frames are extracted per clip). Focusing on concrete, observable non-verbal behaviors relevant to collaborative learning, we investigate how these choices affect behavior detection performance in 360-degree classroom video. Rather than proposing a fully automated solution, this exploratory case study adopts a diagnostic perspective aimed at identifying the conditions under which MLLM-based analysis of 360-degree video is effective or limited. The following research questions guide this study:

RQ1: How does temporal sampling density affect MLLM-based detection of non-verbal behaviors in 360-degree classroom video?

RQ2: How does video extracted view affect MLLM-based detection of non-verbal behaviors?

2. LITERATURE REVIEW

2.1 360-Degree Video in Educational Research

Advances in 360-degree camera technology present new opportunities for using 360 cameras for teaching and learning [41, 17, 35]. 360-degree cameras capture entire learning environments from a single vantage point, recording omnidirectional views that can be navigated from multiple perspectives [17, 35]. Unlike traditional or multi-camera setups that require coordination among multiple devices, 360-degree cameras present explorable spherical scenes that allow researchers and educators to revisit and navigate the learning space, supporting a deeper examination of instructional practices, learners’ engagement, and other social dynamics that occur in the classroom [39, 2]. 360-degree cameras are increasingly used in education for purposes ranging from exam proctoring [1] to immersive training environments, including surgical simulation [24], public speaking practice [44], and observation of teaching practices [48]. However, this flexibility introduces a distinct challenge during video processing. With traditional front-facing cameras, the viewing angle is determined at the time of recording. In contrast, 360-degree video defers this decision to the processing stage, requiring researchers to actively choose how to project and frame the spherical scene for analysis. These decisions are often guided by analytic priorities. Front-facing projections maximize visibility of facial expressions and gaze direction, whereas top-down views better capture spatial arrangements and interactions among group members at a shared workspace [33, 42]. These early choices about viewing angles influence ground-truth annotation and the inputs provided to automated models, making representation selection consequential for human and computational analysis.

These representational decisions compound a broader set of challenges limiting the use of 360-degree video in learning analytics. Processing such footage demands substantial computational resources [32], and standard computer vision models trained on conventional camera footage struggle with 360-degree recordings. Facial, body, and object recognition models often fail due to equirectangular distortion and occlusion common in omnidirectional footage [38]. These challenges point to the need for a systematic evaluation of both video processing methods and computational approaches for collaborative learning analysis.

2.2 Detecting Non-Verbal Behaviors

Early efforts to automate behavior detection have relied on computer vision models trained for specific tasks such as gaze estimation [62], pose detection [55], and gesture recognition [22]. While these approaches have shown promise in controlled settings, they require assembling separate pipelines for each behavior type. For instance, detecting gaze direction may require sequential application of face detection, landmark localization, head pose estimation, and gaze regression, each trained independently and requiring calibration [62]. This presents a substantial engineering burden. A further challenge is that most models are trained on images from standard cameras, assuming consistent scale, frontal viewpoints, and minimal occlusion [57]. As a result, they often struggle to generalize to 360-degree video, where equirectangular projection introduces curvilinear distortion that warps object shapes and spatial relationships [45, 6, 54]. These limitations highlight the need for approaches that can handle the unique visual characteristics of 360-degree footage.

MLLMs offer several potential advantages. First, a single model can describe multiple behavior types without separate pipelines, substantially reducing engineering effort[51]. Second, MLLMs can leverage global context and semantic cues to produce partially grounded interpretations; this context-aware reasoning may support more robust detection of behaviors [60, 56]. Finally, MLLMs produce natural-language outputs that are directly interpretable. But MLLM performance is impacted by how video is represented for input and how temporal information is sampled [26]. Without empirical evidence on how these analytic choices influence behavior detection, it remains unclear when MLLMs can reliably support non-verbal behavior analysis. By identifying which behaviors are robust to variation in representation and temporal sampling, and which are highly sensitive to these choices, such work helps researchers calibrate their validation efforts, anticipating when MLLM outputs are likely to be more or less reliable, and where closer inspection or alternative approaches may be required.

3. METHODS

Figure 2: Example video frames showing non-verbal behaviors in front-face (FF) and top-down (TD) views. Faces and identifying features are obscured for privacy.

3.1 Context and Participants

Data were collected during group activities in an introductory curriculum and instruction course at a Midwestern University. Students engaged with a simulation designed to teach planetary motion and Kepler’s laws (Figure 1), completing a structured group activity involving exploration, data collection, and reflection. A total of 24 out of the 39 students across two classroom sessions consented to participate in the research and worked in groups of two to four. This study focused on one four-student group as a case for examining analytic design choice. The group was selected because the video had high visual quality, all group members remained relatively visible during the focal period, and students worked around a shared workspace with two laptops and worksheets. These features made it possible to compare how the same collaborative activity was represented under different view and sampling conditions. Accordingly, the study should be interpreted as a case-based analysis of preprocessing and detection conditions. Ethical approval was obtained from the institutional review board. All participants provided informed consent for research use of the recordings. Video data were stored and analyzed under approved research procedures.

3.2 Video Data

The selected group’s video lasted approximately 40 minutes. Based on preliminary review, we focused our analysis on the first 20 minutes, after which cross-group interactions became infrequent. Raw .360 files were converted to .mp4 (ProRes) using GoPro Player with H.264 encoding and equirectangular projection. Adobe Premiere Pro v24.1 with a GoPro 360 plugin was then used to extract two viewing conditions (Figure 2). The front-face (FF) view was designed to isolate and maximize the exposure of individual faces to the camera lens, while the top-down (TD) view was designed to isolate and maximize the exposure of the group’s physical interactivity around the table. These two views should not be understood as neutral representations containing identical visual information. Rather, they foreground different aspects of the same 360-degree recording. The FF view provides more direct access to faces and upper-body orientation, whereas the TD view more clearly exposes the table layout, shared artifacts, and relative positioning among students. The comparison therefore examines how view-specific information availability shapes MLLM-based detection. Final videos were rendered at 24 fps and 1080p resolution in .mp4 format.

3.3 Behavior Annotation

Behavior selection was guided by prior work in collaborative learning and non-verbal interaction analysis [14]. The annotation protocol focused on observable, externally visible behaviors, excluding inferences about internal cognitive or affective states. The video was segmented into non-overlapping 10-second clips before annotation. For each clip, trained annotators assigned binary labels indicating whether each behavior in Table 1 was visibly present at any point during the clip. Two trained annotators independently coded the video corpus using the behavior taxonomy summarized in Table 1. Inter-rater agreement was calculated separately for each behavior category using Cohen’s kappa over the 10-second clip-level binary labels. Disagreements were resolved through discussion before producing the final reference labels used for model evaluation. Inter-rater reliability exceeded Cohen’s \(\kappa > 0.75\) for all behavior categories.

3.4 MLLM-Based Behavior Detection

Model and Configuration. We employed Molmo2-8B at full precision (BF16) without quantization. This is an open-weight vision-language model designed for multimodal understanding [12]. Molmo2 integrates a Vision Transformer encoder with a decoder-only language model, using multi-headed attention pooling and an overlapping multi-crop strategy to capture fine-grained visual details with surrounding context. Rather than treating video as an unordered collection of frames, the model associates visual tokens with temporal context, supporting description of actions and events unfolding over short video segments. Molmo2 emphasizes visual grounding through its ability to output 2D points corresponding to pixels supporting its answers. Molmo2-8B was selected for three reasons. First, it is open-weight with a fixed model checkpoint, ensuring exact reproducibility. Second, it achieves benchmark performance comparable to proprietary models such as GPT-4o [12]. Third, its 8B-parameter size allows inference on widely available hardware.The pre-trained model was used without fine-tuning as this work aims to assess baseline model behavior. Temperature was set to zero to ensure deterministic outputs. Inference was conducted on a single NVIDIA H200 GPU.

Video Segmentation. We adopted video-level rather than frame-level behavior detection to support identification of temporal behaviors such as posture changes. To implement this, videos were segmented into fixed-length 10-second clips, a duration that balances temporal context against computational constraints and aligns with the broader multimodal learning analytics project in which this study is situated. Fixed-length segmentation reflects realistic deployment scenarios and avoids reliance on event-based heuristics.

Prompt Design. Each prompt consisted of a system message and a task-specific text prompt. The prompt design draws on prior empirical work showing that explicit role assignment and task framing improve MLLM output quality for educational video analysis [49, 63]. The prompt instructed the model to: "Act as an educational researcher specializing in qualitative and ethnographic video analysis. Your task is to observe the video clip and determine whether the following [BEHAVIOR-CATEGORY]-related behaviors are visually observable. Base your judgment only on visible, externally observable cues, and do not infer attention, intention, engagement, or internal states. If a behavior is ambiguous, partially occluded, or too subtle to identify with confidence, mark it as not detected (0)." This was followed by behavior definitions for each category (see Table 1). Prompts required machine-parsable output and were held constant across all experimental conditions.

Experimental Design. We employed a 2 \(\times \) 4 factorial design, varying video representation (front-facing vs. top-down) and temporal sampling rate (1–4 FPS) while holding model configuration and prompts constant. MLLM outputs were then evaluated against human annotations. Due to space constraints, the main paper reports accuracy and F1 score while full results including precision and recall are provided in the supplementary materials at https://anonymous.4open.science/r/EDM2026-4CC0/. Because the dataset is imbalanced and several behavior categories were relatively infrequent (e.g., gaze away and headnod; Table 1), accuracy and F1 scores should be interpreted with caution. In particular, high accuracy may partly reflect base rates for low-frequency behaviors, while low or unstable F1 scores may reflect data sparsity as well as model difficulty. For this reason, full per-behavior precision and recall results are provided in the supplementary materials.

4. RESULTS

4.1 Effects of Temporal Sampling (RQ1)

To examine how temporal sampling density influenced MLLM-based detection within this specific setup, we compared model performance across four frame rates (1–4 FPS) under both views (see Table 2 and 3). Within the present 10-second clip setup and 1- 4 FPS range, increasing sampling density did not yield consistent improvements in detection performance.

Sustained behaviors (i.e., actions that persist over multiple seconds) demonstrated high and stable detection across all FPS conditions. gaze_at_worksheet maintained strong performance in both views (FF: F1 0.95–0.96; TD: F1 0.93–0.96), as did lean_forward in the TD view (F1 0.92–0.93). Tool usage behaviors all maintained high performance regardless of FPS: usage_of_worksheet (TD: F1 0.88–0.89), usage_of_laptop (TD: F1 0.89–0.90), and sharing_of_laptop (TD: F1 0.89–0.91). These results indicate that for sustained behaviors, lower sampling rates (1–2 FPS) are sufficient. In contrast, several behaviors proved persistently difficult to detect regardless of sampling density. lean_adjacent showed consistently low F1 scores in both views (FF: 0.00–0.17; TD: 0.00–0.06), with accuracy close to the base rate (53%). headnod exhibited similarly poor detection (FF: F1 0.00–0.09; TD: F1 0.00–0.36). Social gaze behaviors (i.e., gaze_at_other and gaze_away) also remained challenging across all sampling rates. For these behaviors, FPS variation did not meaningfully improve performance in this setup, suggesting that sampling density alone may not address detection challenges that arise from relational inference, occlusion, or low behavioral frequency.

Only a small number of behaviors showed sensitivity to sampling density. gesture in the FF view improved modestly (F1 0.12 at 1 FPS to 0.31 at 4 FPS), suggesting that transient hand movements may benefit from denser temporal coverage. However, this pattern did not hold in the TD view, where gesture peaked at 2 FPS (F1 0.39) and declined at higher rates (F1 0.21–0.23). This divergence might point to view-dependent effects on temporal sampling. Other behaviors similarly showed degraded performance at higher rates: lean_forward in the FF view declined from accuracy 0.88 and F1 0.93–0.94 at 1–2 FPS to accuracy 0.69–0.70 and F1 0.80–0.81 at 3–4 FPS. gaze_at_laptop (FF) decreased from F1 0.30 to 0.19 across the FPS range, and sharing_of_laptop (FF) peaked at 2 FPS (F1 0.86) before declining (F1: 0.71–0.73). Together, these findings suggest that uniform increases in temporal sampling do not reliably improve detection performance within this case and sampling range, and that the effects of sampling rate are both behavior- and representation-dependent.

4.2 Effects of Representation View (RQ2)

To examine how video representation influences MLLM-based detection, we compared performance between FF and TD views (see Tables 2 and 3). Performance differed across views, with the TD view generally providing visual information that supported more reliable detection, particularly for behaviors involving shared artifacts and spatial context.

The TD view showed stronger performance for object-related behaviors. gaze_at_laptop showed markedly stronger performance in the TD view (F1 0.84–0.93) compared to the FF view (F1 0.17–0.30). Similar patterns emerged for usage_of_laptop (TD: F1 0.89–0.90 vs. FF: F1 0.59–0.67) and sharing_of_laptop (TD: F1 0.89–0.91 vs. FF: F1 0.71–0.86). These behaviors require information about spatial relationships between learners and artifacts, which was more visible in the global scene layout preserved in top-down projections.

For highly salient sustained behaviors, both views achieved comparable performance. gaze_at_worksheet was detected reliably across representations (FF F1 0.95–0.96; TD F1 0.93–0.96), as was usage_of_worksheet (FF F1 0.89–0.91; TD F1 0.88–0.89). lean_forward performed similarly at lower sampling rates (FF F1 0.93–0.94; TD F1 0.92–0.93), although the FF view showed degradation at higher FPS. These behaviors appear sufficiently visually distinctive that representation choice has limited impact on detection. In contrast, socially oriented behaviors proved difficult to detect under both representations. gaze_at_other yielded low and unstable F1 scores in both views (TD: 0.17–0.40; FF: 0.10–0.23), as did gaze_away (TD: 0.11–0.44; FF: 0.24–0.25). lean_adjacent and headnod remained near zero across all conditions in both views. These behaviors require relational inference about interpersonal coordination, a challenge that neither representation resolved.

5. DISCUSSION

5.1 Temporal Sampling: When Less May Be More

Results for RQ1 show that higher frame sampling rates did not consistently improve detection performance,within the present 10-second clip setup and 1- 4 FPS range. This finding should not be interpreted as a general claim that fewer frames are always preferable. Rather, it suggests that the value of additional frames depends on the temporal characteristics of the target behavior, the length of the clip, and the model’s visual token and context constraints. Within 10-second segments, denser sampling may add redundant frames when behaviors are sustained across multiple seconds. This pattern is consistent with recent findings showing that uniform increases in FPS can plateau or even degrade performance when adjacent frames are highly similar [34, 36]. Many collaborative learning behaviors, such as gazing at a worksheet or maintaining a forward-leaning posture, unfold over several seconds. At higher sampling rates (e.g., 3–4 FPS), the model processes many near-identical frames. This redundancy may dilute the model’s effective attention under fixed token budgets and increase sensitivity to minor visual variations [34]. In contrast, transient behaviors may benefit from higher sampling rates to capture momentary behavioral cues. These findings therefore support behavior-aware sampling rather than uniformly increasing FPS.increasing FPS.

5.2 Affordances for Behavior Detection

Results for RQ2 indicate that video representation impacts which non-verbal behaviors can be reliably detected by video-based MLLMs. A key methodological caveat is that FF and TD views do not contain identical visual information. They are different input conditions that organize visibility in different ways. The TD view more clearly exposes the table layout, shared artifacts, and relative body positions, whereas the FF view provides more direct access to faces and upper-body cues but may obscure the broader workspace. Therefore, the observed differences should be interpreted as view-specific differences in information availability, rather than evidence that the model is inherently better suited to one projection type.

TD views supported more reliable detection of several gaze- and tool-related behaviors, particularly those involving shared artifacts or spatial coordination among learners. This advantage aligns with insights from group activity recognition, which emphasizes that many collaborative behaviors depend on the spatial configuration of multiple participants rather than isolated individual cues [7, 52].

Detecting such behaviors requires knowing whether multiple learners are oriented toward the same object, how close they are to one another, and how their bodies are arranged in space. TD views appear to support these inferences by preserving the global layout of the learning environment, including the relative positions of learners, shared artifacts, and workspaces. By reducing occlusion, normalizing scale differences across learners, and exposing shared reference frames, TD representations foreground scene-level context and inter-person relations that are difficult to reconstruct from frontal viewpoints alone. Similar advantages of global or overhead perspectives have been reported in prior work on collaborative activity recognition, where preserving spatial context improves robustness under occlusion and viewpoint variation [7, 28, 52].

FF views, while less effective overall in our evaluation, have demonstrated strengths in prior work using traditional computer vision approaches. They provide higher resolution of facial features, hand positions, and upper-body movements [4]. Prior research on classroom analytics suggests that such perspectives are well suited for detecting individual-level nonverbal cues, including head orientation, eye gaze, and affect detection [29, 38]. In the present study, however, these advantages did not translate to improved MLLM-based detection as FF views yielded weaker and less stable performance than TD views across most behavior categories. One possible explanation is that MLLMs and traditional computer vision pipelines leverage visual information differently: whereas specialized computer vision models can extract fine-grained facial features from frontal views, MLLMs may rely more heavily on global scene context and semantic cues that are better preserved in TD representations. Additionally, in multi-learner settings, FF views often obscure the broader context needed to infer group-level coordination [31]. Taken together, these findings suggest that no single representation is universally optimal. Representation choice should be guided by analytic goals and methods. Future work should examine whether combining FF and TD information through early or late fusion yields more robust detection [52].

5.3 Object-Related vs. Social Behaviors

One key finding of this study is that object-related behaviors—those involving direct interaction with physical materials such as worksheets and laptops—were detected more reliably than social behaviors requiring inference about interpersonal attention or coordination. For example, higher F1 scores and accuracy were observed for gaze_at_laptop and gaze_at_worksheet compared to gaze_at_other.

Object-related behaviors primarily require perceptual grounding: recognizing the presence of an artifact and identifying that a learner’s gaze or action is directed toward it. Such judgments rely largely on visual pattern matching and benefit from model pretraining on large-scale object recognition data [37]. In contrast, social behaviors require relational inference across participants. Rather than identifying a single target, the model must integrate cues about spatial arrangement, body orientation, and temporal coordination [8, 62]. In 360-degree classroom video, this relational inference is further complicated by wide fields of view and spatial distortion.

These demands remain challenging for current MLLMs, which have been shown to exhibit systematic limitations in interpreting subtle relational aspects of human interaction [30]. These findings have practical implications for the design of MLLM-based analytics pipelines. Object-related behaviors may be more amenable to scalable automated analysis with comparably less human oversight, while social behaviors may require specialized models for detecting interpersonal coordination, such as eye contact and shared attention [18, 46, 28].

5.4 Limitations and Future Work

This study has several limitations. First, This study should be interpreted as a case-based analysis rather than a general evaluation across classroom contexts. The analysis was based on one four-student group, one task, one classroom configuration, and the first 20 minutes of a single 360-degree recording. The selected group was not randomly sampled; it was chosen because the video quality was high and the shared workspace made FF/TD comparisons analytically tractable. As a result, group-level factors such as seating arrangement, communication style, division of labor, technology-use patterns, and visibility of shared artifacts may have influenced the observed differences between FF and TD views. Second, evaluation was conducted using a single MLLM (Molmo2-8B), limiting generalizability to other vision–language models. Differences in architecture, pretraining data, and visual encoding strategies may substantially influence behavior detection performance [9]. Future work should test whether the observed patterns hold across other open-source and proprietary models. Third, the study focused on a specific taxonomy of non-verbal behaviors relevant to collaborative learning. While theoretically grounded, this taxonomy does not exhaustively capture all educationally meaningful behaviors. Third, the study examined only zero-shot inference without fine-tuning or prompt optimization. Although this reflects realistic use cases for many researchers, performance likely underestimates what could be achieved with task-specific adaptation. Techniques such as many-shot in-context learning or chain-of-thought prompting may improve detection accuracy [27, 61].

Beyond these limitations, future research could explore adaptive sampling strategies tailored to specific behavior dynamics, multi-view fusion approaches that leverage complementary perspectives, and fine-tuning methods that adapt pretrained MLLMs to educational contexts. Integrating MLLM-based video analysis with additional data streams—such as audio transcripts, interaction logs, or physiological sensors—also offers promising directions for advancing multimodal learning analytics.

References

1. Y. Atoum, L. Chen, A. X. Liu, S. D. H. Hsu, and X. Liu. Automated online exam proctoring. IEEE Transactions on Multimedia, 19(7):1609–1624, July 2017.
2. N. Balzaretti, A. Ciani, C. Cutting, L. O’Keeffe, and B. White. Unpacking the potential of 360-degree video to support pre-service teacher development. Research on Education and Media, 11(1):63–69, 2019.
3. B. Barron. When smart groups fail. Journal of the Learning Sciences, 12(3):307–359, 2003.
4. A. Behera, P. Matthew, A. Keidel, P. Vangorp, H. Fang, and S. Canning. Associating facial expressions and upper-body gestures with learning tasks for enhancing intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 30(2):236–270, 2020.
5. A. Bewersdorff, C. Hartmann, M. Hornberger, K. Seßler, M. Bannert, E. Kasneci, G. Kasneci, X. Zhai, and C. Nerdel. Taking the next step with generative artificial intelligence: The transformative role of multimodal large language models in science education. Learning and Individual Differences, 118:102601, 2025.
6. M. Cao, C. Mou, F. Yu, X. Wang, Y. Zheng, J. Zhang, and B. Luo. Ntire 2023 challenge on 360\(^\circ \) omnidirectional image and video super-resolution: Datasets, methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1731–1745, 2023.
7. W. Choi and S. Savarese. Understanding collective activities of people from videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(6):1242–1257, 2013.
8. E. Chong, N. Ruiz, Y. Wang, Y. Zhang, A. Rozga, and J. M. Rehg. Connecting gaze, scene, and attention: Generalized attention estimation via joint modeling of gaze and scene saliency. In Proceedings of the European Conference on Computer Vision, pages 383–398, 2018.
9. F. Cocchi, N. Moratelli, D. Caffagni, S. Sarto, L. Baraldi, M. Cornia, and R. Cucchiara. Llava-more: A comparative study of llms and visual backbones for enhanced visual instruction tuning. arXiv preprint, arXiv:2503.15621, 2025.
10. Z. Dai, A. McReynolds, and J. Whitehill. In search of negative moments: Multi-modal analysis of teacher negativity in classroom observation videos. In M. Feng, T. Käser, and P. Talukdar, editors, Proceedings of the 16th International Conference on Educational Data Mining, pages 278–285, Bengaluru, India, July 2023. International Educational Data Mining Society.
11. S. Danish, A. Sadeghi-Niaraki, S. U. R. Khan, L. M. Dang, L. Tightiz, and H. Moon. A comprehensive survey of vision-language models: Pretrained models, fine-tuning, prompt engineering, adapters, and benchmark datasets. Information Fusion, 126:103623, 2025.
12. M. Deitke, C. Clark, S. Lee, R. Tripathi, Y. Yang, J. S. Park, M. Salehi, N. Muennighoff, K. Lo, L. Soldaini, J. Lu, T. Anderson, E. Bransom, K. Ehsani, H. Ngo, Y. Chen, A. Patel, M. Yatskar, C. Callison-Burch, A. Head, R. Hendrix, F. Bastani, E. VanderBilt, N. Lambert, Y. Chou, A. Chheda, J. Sparks, S. Skjonsberg, M. Schmitz, A. Sarnat, B. Bischoff, P. Walsh, C. Newell, P. Wolters, T. Gupta, K.-H. Zeng, J. Borchardt, D. Groeneveld, C. Nam, S. Lebrecht, C. Wittlif, C. Schoenick, O. Michel, R. Krishna, L. Weihs, N. A. Smith, H. Hajishirzi, R. Girshick, A. Farhadi, and A. Kembhavi. Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 91–104, 2025.
13. S. J. Derry, R. D. Pea, B. Barron, R. A. Engle, F. Erickson, R. Goldman, R. Hall, T. Koschmann, J. L. Lemke, and M. G. Sherin. Conducting video research in the learning sciences. Journal of the Learning Sciences, 19(1):3–53, 2010.
14. I. Dey, M. M. Ko, and S. Puntambekar. Visualizing collaboration using multiple modalities. In J. Oshima, B. Chen, F. Vogel, and J. Järvelä, editors, Proceedings of the 18th International Conference on (Computer-Supported) Collaborative Learning - (CSCL) 2025, pages 664–666, 2025.
15. P. Dillenbourg. Collaborative Learning: Cognitive and Computational Approaches. Pergamon, 1999.
16. P. Dillenbourg, M. Baker, A. Blaye, and C. O’Malley. The evolution of research on collaborative learning. In E. Spada and P. Reimann, editors, Learning in Humans and Machine: Towards an Interdisciplinary Learning Science, pages 189–211. Elsevier, Oxford, 1996.
17. M. Evens, M. Empsen, and W. Hustinx. A literature review on 360-degree video as an educational tool. Journal of Computers in Education, 10(2):325–375, 2023.
18. L. Fan, W. Chen, P. Wei, W. Wang, and S.-C. Zhu. Inferring shared attention in social scene videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6460–6468, 2018.
19. S. V. Georgakopoulos, K. Kottari, K. Delibasis, V. P. Plagianakos, and I. Maglogiannis. Pose recognition using convolutional neural networks on omni-directional images. Neurocomputing, 280:23–31, 2018.
20. C. Goodwin. Action and embodiment within situated human interaction. Journal of Pragmatics, 32(10):1489–1522, 2000.
21. M. Holmes, A. Latham, K. Crockett, and J. O’Shea. Near real-time comprehension classification with artificial neural networks. IEEE Transactions on Learning Technologies, 11(1):5–12, 2017.
22. J. Hou, G. Wang, X. Chen, J.-H. Xue, R. Zhu, and H. Yang. Spatial-temporal attention res-tcn for skeleton-based dynamic hand gesture recognition. In Proceedings of the European conference on computer vision (ECCV) workshops, pages 0–0, 2018.
23. B. Huang, X. Wang, H. Chen, Z. Song, and W. Zhu. Vtimellm: Empower llm to grasp video moments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14271–14280, 2024.
24. T. Huber, M. Paschold, C. Hansen, T. Wunderling, H. Lang, and W. Kneist. New dimensions in surgical training: Immersive virtual reality laparoscopic simulation exhilarates surgical staff. Surgical Endoscopy, 31(11):4472–4477, 2017.
25. C. Jewitt, J. Bezemer, and K. O’Halloran. Introducing Multimodality. Routledge, 2016.
26. M. Jia, Z. Qi, S. Zhang, W. Zhang, X. Yu, J. He, and L. Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision–language models. arXiv preprint, arXiv:2506.03135, 2025.
27. Y. Jiang, J. Irvin, J. H. Wang, M. A. Chaudhry, J. H. Chen, and A. Y. Ng. Many-shot in-context learning in multimodal foundation models. arXiv preprint, arXiv:2405.09798, 2024.
28. S. Li, Q. Cao, L. Liu, K. Yang, S. Liu, J. Hou, and S. Yi. Groupformer: Group activity recognition with clustered spatial-temporal transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13668–13677, 2021.
29. F.-C. Lin, H.-H. Ngo, C.-R. Dow, K.-H. Lam, and H.-L. Le. Student behavior recognition system for the classroom environment based on skeleton pose estimation and person detection. Sensors, 21(16):5314, 2021.
30. Z. Liu, P. Rabbani, V. Duddu, K. Fan, M. Lee, and Y. Huang. The social gaze of llms: A literature review of multimodal approaches to human behavior understanding. arXiv preprint, arXiv:2510.23947, 2025.
31. E. A. Mosabbeb, K. Raahemifar, and M. Fathy. Multi-view human activity recognition in distributed camera sensor networks. Sensors, 13(7):8750–8770, 2013.
32. R. Noël, D. Miranda, C. Cechinel, F. Riquelme, T. T. Primo, and R. Munoz. Visualizing collaboration in teamwork: A multimodal learning analytics platform for non-verbal communication. Applied Sciences, 12(15), 2022.
33. S. Oviatt, J. Lin, and A. Sriramulu. I know what you know: What hand movements reveal about domain expertise. ACM Trans. Interact. Intell. Syst., 11(1):1–26, Mar. 2021.
34. J. Park, K. Ranasinghe, K. Kahatapitiya, W. Ryu, D. Kim, and M. S. Ryoo. Too many frames, not all useful: Efficient strategies for long-form video qa. arXiv preprint, arXiv:2406.09396, 2024.
35. J. Qian, J. Shang, and L. Qin. A systematic scoping review of 360-degree videos in teacher education. Journal of Research in Innovative Teaching & Learning, 18(1):20–38, 2025.
36. T. Qu, L. Tang, B. Peng, S. Yang, B. Yu, and J. Jia. Does your vision–language model get lost in the long video sampling dilemma? arXiv preprint, arXiv:2503.12496, 2025.
37. A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, volume 139, pages 8748–8763, 2021.
38. R. J. Rajarathinam, C. Palaguachi, J. Kang, et al. 360-degree cameras vs traditional cameras in multimodal learning analytics: Comparative study of facial recognition and pose estimation. Journal of Educational Data Mining, 17(1):157–182, 2025.
39. J. Reyna. The potential of 360-degree videos for teaching, learning and research. In INTED2018 Proceedings, pages 1448–1454. IATED, 2018.
40. J. Roschelle and S. Teasley. The construction of shared knowledge in collaborative problem solving. In C. O’Malley, editor, Computer-Supported Collaborative Learning, pages 69–97. Springer, 1995.
41. N. L. Schroeder, R. F. Siegle, and S. D. Craig. A meta-analysis on learning from 360° video. Computers & Education, 206:104901, 2023.
42. D. Spikol, E. Ruffaldi, and M. Cukurova. Using multimodal learning analytics to identify aspects of collaboration in project-based learning. In CSCL’17: The 12th International Conference on Computer Supported Collaborative Learning, Philadelphia, PA (June 18-20, 2017), volume 1, pages 263–270. International Society of the Learning Sciences., 2017.
43. G. Stahl. Group Cognition. MIT Press, 2006.
44. S. Stupar-Rutenfrans, L. E. Ketelaars, and M. S. van Gisbergen. Beat the fear of public speaking: Mobile 360 video virtual reality exposure training in home environment reduces public speaking anxiety. Cyberpsychology, Behavior, and Social Networking, 20(10):624–633, 2017.
45. Y.-C. Su and K. Grauman. Learning spherical convolution for fast features from 360 imagery. In Advances in Neural Information Processing Systems, volume 30, pages 529–539, 2017.
46. Ö. Sümer, P. Gerjets, U. Trautwein, and E. Kasneci. Attention flow: End-to-end joint attention estimation. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pages 3316–3325. IEEE, 2020.
47. H. Theelen, A. van den Beemt, and P. den Brok. Using 360-degree videos in teacher education. Journal of Computer Assisted Learning, 35(5):582–594, 2019.
48. N. Walshe and P. Driver. Developing reflective trainee teacher practice with 360-degree video. Teaching and Teacher Education, 78:97–105, 2019.
49. R. Whitehead, A. Nguyen, and S. Järvelä. Utilizing multimodal large language models for video analysis of posture in studying collaborative learning: A case study. Journal of Learning Analytics, 12(1):186–200, 2025.
50. A. F. Wise and B. B. Schwarz. Visions of cscl. International Journal of Computer-Supported Collaborative Learning, 12(4):423–467, 2017.
51. J. Wu, W. Gan, Z. Chen, S. Wan, and P. S. Yu. Multimodal large language models: A survey. In Proceedings of the 2023 IEEE International Conference on Big Data (BigData), pages 2247–2256. IEEE, 2023.
52. L. Wu, Q. Wang, M. Jian, Y. Qiao, and B. Zhao. A comprehensive review of group activity recognition in videos. International Journal of Automation and Computing, 18(3):334–350, 2021.
53. M. Xu, M. Gao, Z. Gan, H.-Y. Chen, Z. Lai, H. Gang, K. Kang, and A. Dehghan. Slowfast-llava: A strong training-free baseline for video large language models. arXiv, 2024. arXiv:2407.15841.
54. Y. Xu, Y. Dong, J. Wu, Z. Sun, Z. Shi, J. Yu, and S. Gao. Gaze prediction in dynamic 360 immersive videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5333–5342. IEEE, 2018.
55. Y. Xu, J. Zhang, Q. Zhang, and D. Tao. Vitpose: Simple vision transformer baselines for human pose estimation. Advances in neural information processing systems, 35:38571–38584, 2022.
56. S. Yin, C. Fu, S. Zhao, K. Li, X. Sun, T. Xu, and E. Chen. A survey on multimodal large language models. National Science Review, 11(12):nwae403, 2024.
57. J. Yu, A. C. P. Grassi, and G. Hirtz. Applications of deep learning for top-view omnidirectional imaging: A survey. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6421–6433, 2023.
58. C. Zahn, A. Ruf, and R. Goldman. Video data collection and video analyses in cscl research. In U. Cress, C. Rosé, A. F. Wise, and J. Oshima, editors, International handbook of computer-supported collaborative learning, pages 643–660. Springer, Cham, 2021.
59. X. Zeng, X. Wang, T. Zhang, C. Yu, S. Zhao, and Y. Chen. Gesturegpt: Toward zero-shot free-form hand gesture understanding with large language model agents. Proceedings of the ACM on Human-Computer Interaction, 8(ISS):462–499, 2024.
60. D. Zhang, X. Zheng, Z. Weng, Y. Lyu, D. P. Paudel, L. Van Gool, and X. Hu. Are multimodal large language models ready for omnidirectional spatial reasoning? arXiv preprint, arXiv:2505.11907, 2025.
61. R. Zhang, B. Zhang, Y. Li, H. Zhang, Z. Sun, Z. Gan, and Y. Yang. Improve vision–language model chain-of-thought reasoning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1631–1662, 2025.
62. Y. Zhong and S. H. Lee. Gazesymcat: A symmetric cross-attention transformer for robust gaze estimation under extreme head poses and gaze variations. Journal of Computational Design and Engineering, 12(3):115–129, 2025.
63. Y. Zhou, J. Kang, and H. Nguyen. Can we trust large language models for video analysis: An exploration of hallucination in multimodal llms. 2025.

© 2026 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.