- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Gaze-based prediction of test performance using machine learning for adaptive learning systems

Yavuz Dinc1,*, Sarah Malone2, Verena Ruf1, Steffen Steinert1, Stefan Küchemann1, and Jochen Kuhn1

  • *Contact author: Dinc.Yavuz@physik.uni-muenchen.de
PDF

Phys. Rev. Phys. Educ. Res. 21, 020148 – Published 26 November, 2025

DOI: https://doi.org/10.1103/4yys-22tc

Abstract

Prior research has demonstrated that students’ performance on physics test items can be accurately predicted using machine learning algorithms based on their gaze behavior. These gaze data are typically recorded during item completion and capture students’ visual attention to both the verbal item stem and accompanying visual representations, such as graphs. This predictive capability highlights the potential of machine learning as a tool for the development of AI-based adaptive learning systems, such as intelligent textbooks, for improving personalized learning. With this study, we aim to replicate the findings on the predictive power of gaze data recorded during test completion in the field of kinematics and extend them by examining whether gaze data collected during a preceding learning phase can similarly predict performance on the test items. The study involved 88 high school students who first engaged with instructional material on kinematics and then completed test items from the test of understanding graphs in kinematics, a test aimed at assessing students’ conceptual understanding of kinematic graphs. Eye-tracking data were recorded both while students engaged with the learning materials and while they worked on the test items presented after each slide. To analyze these data, we used a support vector machine to predict whether students would answer the test items correctly. We compared prediction accuracies based on gaze data recorded during the learning phase with those based on gaze data recorded during the test phase. The findings indicate that eye-tracking data gathered during the completion of test items were more accurate in predicting students’ performance than data collected during the learning phases. Notably, paying close attention to graphs was one of the key behaviors that distinguished students who answered correctly from those who did not. Students’ gaze behavior on the graph was also most predictive of students’ performance. These findings highlight the potential of gaze-based adaptive systems and the importance of including graphical representations in personalized learning environments. They also suggest that future research should explore how intelligent textbooks can combine visual representations, such as graphs, with short conceptual questions to optimize real-time adaptive support.

View figure in article

Physics Subject Headings (PhySH)

Collections

This article appears in the following collection:

Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research

Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research

Article Text

Supplemental Material

References (50)

  1. V. Aleven, I. Roll, B. Mclaren, and K. Koedinger, Help helps, but only so much: Research on help seeking with intelligent tutoring systems, Int. J. Artif. Intell. Educ. 26, 1 (2016).
  2. B. Jiang, M. Gu, and Y. Du, Recent advances in intelligent textbooks for better learning, in AI in Learning: Designing the Future, edited by H. Niemi, R. D. Pea, and Y. Lu (Springer, Cham, 2023), 10.1007/978-3-031-09687-7_15.
  3. S. Järvelä, A. Nguyen, and A. Hadwin, Human and artificial intelligence collaboration for socially shared regulation in learning, Br. J. Educ. Technol. 54, 1057 (2023).
  4. Alexandra Papoutsaki, Patsorn Sangkloy, James Laskey, Nediyana Daskalova, Jeff Huang, and James Hays, WebGazer: Scalable webcam eye tracking using user interactions, in Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI’16) (AAAI Press, New York, 2016), pp. 3839–3845.
  5. Kyle Krafka, Aditya Khosla, Petr Kellnhofer, Harini Kannan, Suchendra Bhandarkar, Wojciech Matusik, and Antonio Torralba, Eye tracking for everyone, in Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV (IEEE, New York, 2016).
  6. Miriam Navarro, Álvaro Becerra, Roberto Daza, Ruth Cobos, Aythami Morales, and Julian Fierrez, VAAD: Visual attention analysis dashboard applied to e-Learning, in Proceedings of the 2024 International Symposium on Computers in Education, SIIE, A Coruña, Spain (IEEE, New York, 2024).
  7. A. R. Strohmaier, K. J. MacKay, A. Obersteiner, and K. M. Reiss, Eye-tracking methodology in mathematics education research: A systematic literature review, Educ. Stud. Math. 104, 147 (2020).
  8. E. Alemdag and K. Cagiltay, A systematic review of eye tracking research on multimedia learning, Comput. Educ. 125, 413 (2018).
  9. K. Scheiter, C. Schubert, A. Schüler, H. Schmidt, G. Zimmermann, B. Wassermann, and T. Eder, Adaptive multimedia: Using gaze-contingent instructional guidance to provide personalized processing support, Comput. Educ. 139, 31 (2019).
  10. S. Becker, S. Küchemann, P. Klein, A. Lichtenberger, and J. Kuhn, Gaze patterns enhance response prediction: More than correct or incorrect, Phys. Rev. Phys. Educ. Res. 18, 020107 (2022).
  11. S. Küchemann, P. Klein, S. Becker, N. Kumari, and J. Kuhn, Classification of students’ conceptual understanding in STEM education using their visual attention distributions: A comparison of three machine-learning approaches, in Proceedings of the 12th International Conference on Computer Supported Education—Volume 1: CSEDU (SciTePress, 2020), pp. 36–46, ISBN [Amazon][WorldCat], 10.5220/0009359400360046.
  12. V. Ruf, A. Horrer, M. Berndt, S. I. Hofer, F. Fischer, M. R. Fischer, J. M. Zottmann, J. Kuhn, and S. Küchemann, A literature review comparing experts’ and non-experts’ visual processing of graphs during problem-solving and learning, Educ. Sci. 13, 216 (2023).
  13. D. Dzsotjan, K. Ludwig-Petsch, S. Mukhametov, S. Ishimaru, S. Kuechemann, and J. Kuhn, The predictive power of eye-tracking data in an interactive AR learning environment, in Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the Proceedings of the 2021 ACM International Symposium on Wearable Computers (Association for Computing Machinery, New York, NY, 2021), pp. 467–471, 10.1145/3460418.3479358.
  14. P. Klein, J. Viiri, S. Mozaffari, A. Dengel, and J. Kuhn, Instruction-based clinical eye-tracking study on the visual interpretation of divergence: How do students look at vector field plots?, Phys. Rev. Phys. Educ. Res. 14, 010116 (2018).
  15. W. M. Roth and K. Tobin, Cascades of inscriptions and the re-presentation of nature: How numbers, tables, graphs, and money come to re-present a rolling ball, Int. J. Sci. Educ. 19, 1075 (1997).
  16. S. Ainsworth, DeFT: A conceptual framework for considering learning with multiple representations, Learn. Instr. 16, 183 (2006).
  17. K. R. Butcher, The multimedia principle, in The Cambridge Handbook of Multimedia Learning (2014), pp. 174–205, 10.1017/cbo9781139547369.010.
  18. S. Ainsworth and A. T. Loizou, The effects of self-explaining when learning with text or diagrams, Cognit. Sci. 27, 669 (2003).
  19. W. Schnotz, Integrated model of text and picture comprehension, in The Cambridge Handbook of Multimedia Learning, 2nd ed., edited by R. E. Mayer (Cambridge University Press, Cambridge, England, 2014), pp. 72–103, 10.1017/CBO9781139547369.006.
  20. S. Ainsworth, The multiple representation principle in multimedia learning, in The Cambridge Handbook of Multimedia Learning, 2nd ed., edited by R. E. Mayer (Cambridge University Press, Cambridge, England, 2014), pp. 464–486, 10.1017/CBO9781139547369.024.
  21. E. Rexigel, J. Kuhn, S. Becker, and S. Malone, The more the better? A systematic review and meta-analysis of the benefits of more than two external representations in STEM education, Educ. Psychol. Rev. 36, 124 (2024).
  22. S. Malone, K. Altmeyer, M. Vogel, and R. Brünken, Homogeneous and heterogeneous multiple representations in equation-solving problems: An eye-tracking study, J. Comput. Assist. Learn. 36, 781 (2020).
  23. N. Ott, R. Brünken, M. Vogel, and S. Malone, Multiple symbolic representations: The combination of formula and text supports problem solving in the mathematical field of propositional logic, Learn. Instr. 58, 88 (2018).
  24. Marlit Annalena Lindner, Alexander Eitel, Benjamin Strobel, and Olaf Köller, Identifying processes underlying the multimedia effect in testing: An eye-movement analysis, Learn. Instr. 47, 91 (2017).
  25. R. E. Mayer, Cognitive theory of multimedia learning, in The Cambridge Handbook of Multimedia Learning (2014), pp. 43–71, 10.1017/cbo9781139547369.005.
  26. M. A. Lindner, A. Eitel, J. Barenthien, and O. Köller, An integrative study on learning and testing with multimedia: Effects on students’ performance and metacognition, Learn. Instr. 71, 101100 (2021).
  27. S. Malone, K. Altmeyer, M. Vogel, and R. Brünken, Homogeneous and heterogeneous multiple representations in equation-solving problems: An eye-tracking study, J. Comput. Assist. Learn. 36, 781 (2020).
  28. M. A. Lindner, A. Eitel, B. Strobel, and O. Köller, Identifying processes underlying the multimedia effect in testing: An eye-movement analysis, Learn. Instr. 47, 91 (2017).
  29. P. Klein, S. Becker, S. Kuchemann, and J. Kuhn, Test of understanding graphs in kinematics: Item objectives confirmed by clustering eye movement transitions, Phys. Rev. Phys. Educ. Res. 17, 013102 (2021).
  30. L. Hahn and P. Klein, Eye tracking in physics education research: A systematic literature review, Phys. Rev. Phys. Educ. Res. 18, 013102 (2022).
  31. H. Haider and P. A. Frensch, Eye movement during skill acquisition: More evidence for the information-reduction hypothesis, J. Exp. Psychol. Learn. Mem. Cogn. 25, 172 (1999).
  32. Q. Wang, Y. Zhu, L. Wei, and H. Deng, Visual attention pattern of middle school students during problem-solving in physics, Mind Brain Educ. 16, 99 (2022).
  33. V. Ruf, Y. Dinc, S. Küchemann, M. Berndt, S. Steinert, D. Kugelmann, and J. Kuhn, Comparison and AI-based prediction of graph comprehension skills based on the visual strategies of first-year physics and medicine students, Phys. Rev. Phys. Educ. Res. 20, 020138 (2024).
  34. R. J. Beichner, Testing student interpretation of kinematics graphs, Am. J. Phys. 62, 750 (1994).
  35. Géron, Hands-On Machine Learning with scikit-learn, keras, and tensorflow, 2nd ed. (O’Reilly Media Inc., Canada, 2019).
  36. S. Mozaffari, P. Klein, M. Al-Naser, S. Küchemann, J. Kuhn, T. Widmann, and A. Dengel, Quantifying gaze-based strategic patterns in physics vector field divergence, in Proceedings of the Agents and Artificial Intelligence: 12th International Conference, ICAART 2020, Valletta, Malta, February 22–24, 2020, Revised Selected Papers 12 (Springer International Publishing, New York, 2021), pp. 465–481.
  37. D. Ifenthaler and C. Widanapathirana, Development and validation of a learning analytics framework: Two case studies using support vector machines, Technol. Knowl. Learn. 19, 221 (2014).
  38. C. Verma, V. Stoffová, and Z. Illés, Prediction of students’ awareness level towards ICT and mobile technology in Indian and Hungarian University for the real-time: Pre-liminary results, Heliyon 5, e01806 (2019).
  39. T. Kabudi, I. Pappas, and D. H. Olsen, AI-enabled adaptive learning systems: A systematic mapping of the literature, Comput. Educ. 2, 100017 (2021).
  40. A. Namoun and A. Alshanqiti, Predicting student performance using data mining and learning analytics techniques: A systematic literature review, Appl. Sci. 11, 237 (2020).
  41. Tobii Pro Spark, eye tracker, https://www.tobii.com/products/eye-trackers/screen-based/tobii-pro-spark [accessed December 7, 2024]
  42. D. D. Salvucci and J. H. Goldberg, Identifying fixations and saccades in eye-tracking protocols, in Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (Association for Computing Machinery, New York, NY, 2000), pp. 71–78.
  43. See Supplemental Material at http://link.aps.org/supplemental/10.1103/4yys-22tc for learning material (German).
  44. P. Klein, A. Lichtenberger, S. Küchemann, S. Becker, M. Kekule, J. Viiri, C. Baadte, A. Vaterlaus, and J. Kuhn, Visual attention while solving the test of understanding graphs in kinematics: An eye-tracking analysis, Eur. J. Phys. 41, 025701 (2020).
  45. K. Wada, labelme: Image polygonal annotation with python [Computer software], 10.5281/zenodo.5711226.
  46. S. Steinert, V. Ruf, D. Dzsotjan, N. Großmann, A. Schmidt, J. Kuhn, S. Küchemann, and S. Veerappampalayam Easwaramoorthy, A refined approach for evaluating small datasets via binary classification using machine learning, PLoS One 19, e0301276 (2024).
  47. Kathrin Kennel, Shoya Ishimaru, Stefan Küchemann, Steffen Steinert, Jochen Kuhn, and Stefan Ruzika, Gaze-based prediction of students’ math difficulties—A time dynamic machine learning approach to enable early individual assistance, Int. J. Artif. Intell. Educ. (2025), 10.1007/s40593-024-00447-5.
  48. Saleh Mozafari, Mohammad Al-Naser, Pascal Klein, Stefan Küchemann, Jochen Kuhn, Thomas Widmann, and Andreas Dengel, Classification of Visual Strategies in Physics Vector Field Problem-Solving (2020), pp. 257–267, 10.5220/0009173902570267.
  49. N. Rebello, M. Nguyen, Y. Wang, T. Zu, J. Hutson, and L. Loschky, Machine learning predicts responses to conceptual tasks using eye movements, presented at PER Conf. 2018, Washington, DC, 10.1119/perc.2018.pr.Rebello.
  50. M. Sajka and R. Rosiek, Analysis of aspects of visual attention when solving multiple-choice science problems, in Applying Bio-Measurements Methodologies in Science Education Research, edited by I. Devetak and S. A. Glažar (2021), pp. 185–215, 10.1007/978-3-030-71535-9_10.

Outline

Information

Phys. Rev. Phys. Educ. Res. 21, 020148– Published 26 November, 2025
  • Received 15 December 2024
  • Accepted 9 October 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation