ABSTRACT

When a bilingual second-grader describes a triangle as “pizza” on a geometry assessment, current AI tutoring systems face a classification dilemma: is this a conceptual error, or evidence of a student using everyday vocabulary (Basic Interpersonal Communicative Skills) because they lack the formal mathematical register (Cognitive Academic Language Proficiency)? No existing educational AI system separates mathematical conceptual understanding from second-language processing at the individual response level. We present CL-KT-X, a neuro-symbolic diagnostic engine that performs cross-lingual knowledge tracing through a three-layer architecture: deterministic linguistic fingerprinting, EXTRACT-NORMALIZE-COMPARE algorithmic prompting, and a Truth-First classification router. In validating CL-KT-X, we document a previously unnamed phenomenon—the AI Whisperer Effect—in which architecturally similar local LLMs require model-specific prompt engineering to achieve comparable diagnostic accuracy. Validated on 74 authentic responses from bilingual Title I classrooms across two mathematical domains, the system achieves 98.6% classification accuracy with zero Truth-First violations. This work crosses linguistic, disciplinary, and contextual borders to address equity in AI-assisted mathematics assessment for the 5.3 million emergent bilingual students in U.S. schools.

Keywords

1. INTRODUCTION

Emergent bilingual students—those developing proficiency in English while maintaining their native language—navigate two linguistic systems simultaneously when responding to mathematical assessments. A student who writes “pizza” for triangle is not demonstrating a geometric misconception; she is applying a Basic Interpersonal Communicative Skills (BICS) analogy because she has not yet acquired the Cognitive Academic Language Proficiency (CALP) term in her second language [4, 5]. Current AI-powered assessment systems cannot make this distinction, systematically misclassifying linguistically mediated responses as conceptual errors.

The consequences are measurable. More than 5.3 million emergent bilingual students attend U.S. public schools, with over one million in Grades 2–3 [8]. English learners are identified for special education at elevated rates in unadjusted models (OR \(\approx \) 2.56 at Grade 3), attenuating under full covariate adjustment [9]; among those referred for reading concerns, most did not qualify for the referred disability on evaluation (34 of 44) [10]. Traditional knowledge tracing approaches model mastery as a single latent variable [3, 11, 6], capturing whether a student errs but not why. They cannot distinguish “doesn’t understand circles” from “understands circles but calls them círculos.”

We present CL-KT-X (Cross-Lingual Knowledge Tracing with Explainable Feedback), a neuro-symbolic diagnostic engine that separates L1 conceptual understanding from L2 language processing by routing student responses through a deterministic pipeline where only the extraction phase is LLM-dependent. Our contributions are threefold:

1. A neuro-symbolic architecture for cross-lingual mathematical diagnosis, combining deterministic linguistic fingerprinting with constrained LLM extraction and an invariant Truth-First classification router.
2. Authentic validation on 74 classroom responses from bilingual Title I students across two mathematical domains, achieving 98.6% classification accuracy—eliminating reliance on synthetic test batteries.
3. Documentation of the AI Whisperer Effect, a previously unnamed phenomenon in which architecturally similar sub-10B parameter LLMs exhibit divergent diagnostic accuracy under identical algorithmic prompts.

This work responds directly to the EDM 2026 theme of Educational Data Mining Across Borders. CL-KT-X crosses three borders simultaneously: the language border between Spanish and English student responses, the discipline border integrating computational linguistics with mathematics education and knowledge tracing, and the context border between everyday communicative register (BICS) and academic assessment vocabulary (CALP).

2. RELATED WORK

Knowledge tracing has evolved from Bayesian Knowledge Tracing [3] through deep neural architectures [11, 13] to context-aware attentive models [6]. These advances have substantially improved prediction of student performance trajectories, yet they share a structural limitation: knowledge is modeled as a single latent construct. A student transitioning from “unlearned” to “learned” on a fractions concept may have always understood the mathematics but lacked the second-language vocabulary to express it. To the authors’ knowledge, no existing knowledge tracing model separates conceptual from linguistic knowledge states.

Bilingual education research has long recognized the challenge of disentangling language proficiency from content mastery [1, 12]. Cummins’ BICS/CALP framework [4, 5] establishes that conversational fluency (BICS) is acquired in 1–3 years, while academic register (CALP) requires 5–7 years. In mathematics, a student may operate at CALP level in Spanish but BICS level in English, producing responses that appear mathematically incorrect under monolingual evaluation. Accommodations such as translated assessments improve validity [12] but do not address real-time diagnostic classification at the individual response level.

The application of large language models to educational assessment has expanded rapidly [7], yet these systems treat language as a transparent medium rather than a variable that interacts with domain knowledge. The emerging field of algorithmic prompting—providing LLMs with step-by-step computational procedures rather than task descriptions—has demonstrated gains in mathematical reasoning [14] but has not been applied to cross-lingual diagnostic contexts. Furthermore, the assumption that prompt engineering transfers uniformly across models within the same parameter class has only recently begun to be examined. Our work addresses both gaps: we apply algorithmic prompting to cross-lingual diagnosis and document empirically that this assumption does not hold.

3. SYSTEM ARCHITECTURE

CL-KT-X employs a three-layer neuro-symbolic architecture. Layer 1: Linguistic Fingerprint—a deterministic module that detects Spanish-origin tokens, phonetic approximations (e.g., “tree” for three), and BICS vocabulary using curated lookup tables, requiring no LLM invocation. Layer 2: ENC Algorithmic Prompting—the LLM performs only EXTRACT (identifying the student’s mathematical claim from a free-text response); NORMALIZE, COMPARE, and CLASSIFY execute deterministically in Python using curated BICS-to-CALP equivalence tables. Layer 3: Truth-First Router—an architectural invariant ensuring that mathematically incorrect answers always resolve to Conceptual_Gap regardless of detected language artifacts.

Algorithm 1 presents the ENC pipeline. The critical design decision is the separation of EXTRACT (LLM-dependent) from NORMALIZE-COMPARE-CLASSIFY (deterministic). This constrains LLM hallucination to the extraction phase, where it is intercepted by the normalizer’s curated lookup tables before classification. The Truth-First Router (line 5) cannot be overridden by language detection.

4. METHODOLOGY

4.1 Corpus

The validation corpus comprises 74 authentic student responses collected from bilingual Title I elementary classrooms in the [REDACTED] metropolitan area. Two mathematical domains are represented: Geometry (37 vectors) and Fractions (37 vectors). All responses are unedited, raw classroom artifacts produced by emergent bilingual students (Grades 2–5) during regular instructional assessment. No synthetic augmentation or researcher-generated examples were used. To the authors’ knowledge, this represents the first authentic cross-lingual mathematical response dataset used to validate an AI diagnostic system.

Response collection involved de-identified archival artifacts from routine instructional assessment conducted by the Co-PI in their capacity as a certified classroom educator. No personally identifiable information was collected or retained. Student data was handled under FERPA-compliant local processing protocols.

4.2 Dual-Review Protocol

Each response underwent a dual-review classification protocol. First, an independent pedagogical classification by a certified bilingual education specialist (15 years of classroom experience, campus-level teaching recognition) assigned ground-truth labels (Conceptual_Gap, Linguistic_Gap, or Correct) based on professional assessment of the student’s mathematical understanding and language use. Second, blind CL-KT-X engine processing produced classifications with no access to the specialist’s labels. Discrepancies were reconciled by a certified bilingual educator serving as Principal Investigator. The PI override rate of 17.1% across all domains documents productive human-AI collaboration in the diagnostic loop—evidence that the system augments rather than replaces expert judgment.

Gold-standard labels for the expanded corpus (\(N=97\), math and geometry domains, SO-15-01 stratum filter applied) were established through joint expert consensus scoring by the two authors, both certified bilingual educators with extensive TELPAS calibration experience (TEA official records). Labels were assigned collaboratively through expert deliberation rather than independent blind annotation; this joint consensus process constitutes the gold standard against which CL-KT-X is evaluated. Human-system agreement between this gold standard and CL-KT-X output was assessed using Cohen’s \(\kappa \), yielding \(\kappa = 0.8785\) (95% CI [0.790, 0.954], BCa bootstrap, \(B = 20{,}000\) iterations), exceeding the \(\kappa \geq 0.80\) threshold established as the minimum for reliable NLP annotation [2]. This metric reflects agreement between the joint expert consensus gold standard and the system output; it is not a measure of independent inter-annotator agreement. Independent multi-annotator validation is identified as future work (Section 6.2).

4.3 Model Configuration

CL-KT-X was validated using three local LLMs deployed via Ollama: Llama 3 (8B parameters), Mistral 7B-Instruct, and Qwen 2.5 (7B parameters). All models received identical ENC algorithmic prompts. Local deployment ensures FERPA compliance—no student data leaves the school network. The deliberate selection of architecturally similar sub-10B parameter models serves a dual purpose: it tests whether the ENC framework enables diagnostic accuracy with resource-constrained models deployable in Title I school infrastructure, and it creates the controlled conditions under which the AI Whisperer Effect becomes observable.

5. RESULTS

5.1 Domain Certification

Table 1 presents classification accuracy across two mathematical domains. Geometry achieved 36/37 correct classifications (97.3%) and Fractions achieved 37/37 (100.0%), yielding a combined accuracy of 73/74 (98.6%) across all authenticated vectors. Zero Truth-First violations were recorded across both domains—no mathematically incorrect response was classified as a linguistic gap. The single geometry misclassification involved a multi-claim response where the LLM extraction phase truncated a semantically complex student utterance, a known boundary condition documented as the BICS Semantic Ceiling.

5.2 The AI Whisperer Effect

Table 2 reveals a previously undocumented phenomenon we term the AI Whisperer Effect. Across three architecturally similar local LLMs and three adversarial prompting strategies applied to the same \(N=97\) corpus, unweighted \(\kappa \) ranges from \(-0.0047\) (Llama 3, Role Injection) to \(+0.2449\) (Mistral, Role Injection)—a spread of more than 0.25 \(\kappa \) units among models comparable in parameter count (7–8B) and architecture family. No LLM cell exceeds \(\kappa = 0.25\) on either metric; all remain far below the XODA baseline of \(\kappa = 0.8785\). These models respond differently to identical algorithmic scaffolding, and no prompting strategy rescues performance.

This finding challenges the assumption that prompt engineering transfers across models within the same parameter class. We hypothesize that the AI Whisperer Effect emerges from differences in instruction-tuning procedures and tokenizer behavior that manifest specifically under the constrained extraction tasks required by cross-lingual diagnosis. The practical implication is significant: deployed educational AI systems using local LLMs require empirical validation of prompt–model compatibility, not architectural assumption.

5.3 Prompt Evolution

The 98.6% combined accuracy with Llama 3 represents a substantial improvement over the 64.9% baseline achieved with the initial prompt version on the same fractions corpus—a progression spanning four major revisions. Notably, an intermediate revision that improved geometry performance caused a regression in fractions accuracy (94.6% \(\to \) 83.8%), demonstrating that prompt modifications for one mathematical domain can cascade into regressions in another. This Prompt Example Cascade is a subtype of the AI Whisperer Effect under investigation in ongoing work.

6. DISCUSSION AND CONCLUSION

6.1 Implications

CL-KT-X demonstrates that cross-lingual knowledge tracing is achievable with resource-constrained local LLMs when the system architecture separates LLM-dependent extraction from deterministic classification. The neuro-symbolic design constrains hallucination risk to a single pipeline stage, making the system’s reasoning transparent and auditable—properties essential for high-stakes educational assessment. The Truth-First Router provides a hard guarantee that no mathematically incorrect response can be misclassified as a language barrier, an invariant that purely neural approaches cannot offer.

The AI Whisperer Effect has implications beyond bilingual education. Any deployed system using local LLMs for student response analysis must account for model-specific prompt sensitivity, even among architecturally similar models. This finding reframes what might be dismissed as “brittleness” as a systematic property of small-model behavior under algorithmic prompting constraints.

6.2 Limitations

Five limitations warrant explicit disclosure. First, CL-KT-X was evaluated against the same 97-item corpus used during system development; this in-sample evaluation means the \(\kappa = 0.8785\) figure reflects performance under favorable conditions. The LLMs reported in Table 2 were evaluated zero-shot on the same items, creating an asymmetric comparison (in-sample system versus zero-shot baselines). This asymmetry does not invalidate the comparison—both XODA and the LLMs were scored against the same joint expert consensus gold standard using the same metric—but readers should interpret the \(\Delta \) advantage accordingly. Independent held-out evaluation on unseen data is the primary objective of ongoing work. Second, gold-standard labels were established through joint expert consensus by the two co-authors; as the authors also developed the system being evaluated, an inherent conflict of interest exists. Independent blind annotation by researchers unaffiliated with system development is identified as a necessary condition for broader validity claims and is planned as the next annotation phase. Third, the validation corpus of 74 vectors across two domains, while authentic, is modest in scale; expansion to additional domains (decimals, word problems) and grade levels is in progress. Fourth, the AI Whisperer Effect has been documented across three models and four prompting strategies; broader surveys are needed to characterize the phenomenon fully. Fifth, the current system targets mathematics; whether the ENC framework generalizes to other content areas (e.g., science vocabulary) remains an open question. These represent active research directions rather than architectural constraints.

6.3 Conclusion

This work makes three contributions to educational data mining across borders: a neuro-symbolic architecture for cross-lingual mathematical diagnosis, an authentic validation protocol grounded in bilingual classroom practice, and the identification of the AI Whisperer Effect as a systematic phenomenon in local LLM deployment for educational assessment. Future work includes domain expansion, longitudinal tracking of student knowledge states across assessments, and systematic investigation of the Prompt Example Cascade. CL-KT-X represents a step toward equitable AI-assisted assessment for the 5.3 million emergent bilingual students whose mathematical understanding currently risks being lost in translation.

7. ACKNOWLEDGMENTS

The authors thank the bilingual Title I classroom educators whose instructional practice produced the authentic response artifacts that ground this research. This work is conducted independently by mar&ar ideas products &more, LLC / QuizMyBrainz.

8. REFERENCES

1. J. Abedi. The No Child Left Behind Act and English language learners: Assessment and accountability issues. Educational Researcher, 33(1):4–14, 2004.
2. R. Artstein and M. Poesio. Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596, 2008.
3. A. T. Corbett and J. R. Anderson. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4):253–278, 1995.
4. J. Cummins. Cognitive/academic language proficiency, linguistic interdependence, the optimum age question and some other matters. Working Papers on Bilingualism, 19:121–129, 1979.
5. J. Cummins. Age on arrival and immigrant second language learning in Canada: A reassessment. Applied Linguistics, 2(2):132–149, 1981.
6. A. Ghosh, N. Heffernan, and A. S. Lan. Context-aware attentive knowledge tracing. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery /span> Data Mining, pages 2330–2339, 2020. [7] E. Kasneci, K. Seßler, S. Küchemann, et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103:102274, 2023. [8] National Center for Education Statistics. English learners in public schools. U.S. Department of Education, 2024. https://nces.ed.gov/programs/coe/indicator/cgf [9] S. O. Ortiz and B. Y. Chow. Disproportionate identification of English learners with specific learning disabilities: Examining grade-level odds and covariate adjustment. Research in Special Education, 3, 2026. https://doi.org/10.25894/rise.2785 [10] S. O. Ortiz, M. Fránquiz, and A. Lara. Disentangling language from learning disability: Evaluation outcomes for emergent bilinguals referred for reading concerns. Bilingual Research Journal, 43(3):245–252, 2020. https://doi.org/10.1080/15235882.2020.1823734 [11] C. Piech, J. Bassen, J. Huang, et al. Deep knowledge tracing. Advances in Neural Information Processing Systems, 28, 2015. [12] G. Solano-Flores. Who is given tests in what language by whom, when, and where? The need for probabilistic views of language in the testing of English language learners. Educational Researcher, 37(4):189–199, 2008. [13] J. Zhang, X. Shi, I. King, and D.-Y. Yeung. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th International Conference on World Wide Web, pages 765–774, 2017. [14] D. Zhou, N. Schärli, L. Hou, et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625, 2022.

© 2026 Copyright is held by the author(s). This work is distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.