- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Dual-diagnostic approach for jointly estimating students’ conceptual understanding and misconceptions in forces and motion

Kuerbanjiang Jielili1,2, Ziqi Yao1,2, Shuhe Zhang1,2, Mingpu Lan3, and Jianxin Yao1,2,*

  • 1School of Physics and Astronomy, Beijing Normal University, Beijing 100875, China
  • 2Key Laboratory of Multiscale Spin Physics (Ministry of Education), Beijing Normal University, Beijing 100875, China
  • 3Dongguan High School, Dongguan, Guangdong 523000, China

  • *Contact author: yaojianxin@bnu.edu.cn
PDF

Phys. Rev. Phys. Educ. Res. 22, 010133 – Published 9 April, 2026

DOI: https://doi.org/10.1103/hrx4-gpd2

Abstract

Diagnosing students’ mastery of physics concepts alongside their misconceptions is important for guiding effective physics instruction. Prior studies have analyzed distractor choices to identify misconceptions, and recent research has applied cognitive diagnostic modeling to obtain more fine-grained insights into student understanding. Building on this progress, we first proposed the two-stage multiple-choice deterministic inputs, noisy “and” gate with hierarchy (TS-MC-DINA-H) model, which estimates the probabilities of both conceptual mastery and misconceptions through option-level response coding. On the basis of this model, we developed a cognitive diagnostic assessment tool for forces and motion. An analysis of data from 492 high school students validated the assessment tool. Moreover, the diagnostic results revealed that students with identical total scores often exhibited different mastery-misconception profiles, thus suggesting that diagnostic information can complement traditional score-based evaluation. The key goal of this study is to provide a new approach for simultaneously diagnosing concept mastery and a wide range of misconceptions at the individual level. This dual diagnostic perspective provides a more comprehensive picture of student learning and offers practical implications for targeted instruction and personalized remediation.

View figure in article

Physics Subject Headings (PhySH)

Article Text

Supplemental Material

References (93)

  1. Ministry of Education of the People’s Republic of China, General Senior Secondary Physics Curriculum Standards, 2017 ed., revised 2018 (People’s Education Press, Beijing, 2018) (in Chinese).
  2. NGSS Lead States, Next Generation Science Standards: For States, By States (The National Academies Press, Washington, DC, 2013).
  3. J. Clement, Students’ preconceptions in introductory mechanics, Am. J. Phys. 50, 66 (1982).
  4. R. F. Gunstone and R. T. White, Understanding of gravity, Sci. Educ. 65, 291 (1981).
  5. I. A. Halloun and D. Hestenes, The initial knowledge state of college physics students, Am. J. Phys. 53, 1043 (1985).
  6. I. A. Halloun and D. Hestenes, Common sense concepts about motion, Am. J. Phys. 53, 1056 (1985).
  7. M. McCloskey, A. Caramazza, and B. Green, Curvilinear motion in the absence of external forces: Naïve beliefs about the motion of objects, Science 210, 1139 (1980).
  8. M. McCloskey, A. Washburn, and L. Felch, Intuitive physics: The straight-down belief and its origin, J. Exp. Psychol. Learn. Mem. Cognit. 9, 636 (1983).
  9. G. J. Posner, K. A. Strike, P. W. Hewson, and W. A. Gertzog, Accommodation of a scientific conception: Toward a theory of conceptual change, Sci. Educ. 66, 211 (1982).
  10. L. C. McDermott, Millikan lecture 1990: What we teach and what is learned—Closing the gap, Am. J. Phys. 59, 301 (1991).
  11. National Research Council, How People Learn: Brain, Mind, Experience, and School, Expanded ed. (National Academies Press, Washington, DC, 2000).
  12. J. P. Leighton and M. J. Gierl, Cognitive Diagnostic Assessment for Education: Theory and Applications (Cambridge University Press, Cambridge, 2007).
  13. A. A. Rupp, J. Templin, and R. A. Henson, Diagnostic Measurement: Theory, Methods, and Applications (Guilford Press, New York, 2010).
  14. M. J. Gierl, J. P. Leighton, and S. M. Hunka, An NCME instructional module on exploring the logic of Tatsuoka’s rule-space model for test development and analysis, Educ. Meas. 19, 34 (2000).
  15. K. K. Tatsuoka, Rule space: An approach for dealing with misconceptions based on item response theory, J. Educ. Meas. 20, 345 (1983).
  16. L. V. DiBello, L. A. Roussos, and W. F. Stout, Review of cognitively diagnostic assessment and a summary of psychometric models, in Handbook of Statistics, Vol. 26: Psychometrics, edited by C. R. Rao and S. Sinharay (Elsevier, Amsterdam, 2007), pp. 979–1030.
  17. A. A. Rupp and J. L. Templin, Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art, Measurement 6, 219 (2008).
  18. J. de la Torre, DINA model and parameter estimation: A didactic, J. Educ. Behav. Stat. 34, 115 (2009).
  19. J. L. Templin and R. A. Henson, Measurement of psychological disorders using cognitive diagnosis models, Psychol. Methods 11, 287 (2006).
  20. J. de la Torre, The generalized DINA model framework, Psychometrika 76, 179 (2011).
  21. T. Xu, X. Wu, S. Sun, and Q. Kong, Cognitive diagnostic analysis of students’ mathematical competency based on the DINA model, Psychologie 60, 3135 (2023).
  22. Y. Dong, X. Ma, C. Wang, and X. Gao, An optimal choice of cognitive diagnostic model for second language listening comprehension test, Front. Psychol. 12, 608320 (2021).
  23. B. C. Kuo, C. H. Chen, C. W. Yang, and M. M. C. Mok, Cognitive diagnostic models for tests with multiple-choice and constructed-response items, Educ. Psychol. 36, 1115 (2016).
  24. B. C. Kuo, C. H. Chen, and J. de la Torre, A cognitive diagnosis model for identifying coexisting skills and misconceptions, Appl. Psychol. Meas. 42, 179 (2018).
  25. D. Hestenes, M. Wells, and G. Swackhamer, Force Concept Inventory, Phys. Teach. 30, 141 (1992).
  26. R. Driver, A. Squires, P. Rushworth, and V. Wood-Robinson, Making Sense of Secondary Science: Research into Children’s Ideas (Routledge, London, 1994).
  27. R. K. Thornton and D. R. Sokoloff, Assessing student learning of Newton’s laws: The force and motion conceptual evaluation and the evaluation of active-learning laboratory and lecture curricula, Am. J. Phys. 66, 338 (1998).
  28. L. Bao and E. F. Redish, Concentration analysis: A quantitative assessment of student states, Am. J. Phys. 69, S45 (2001).
  29. T. Martin-Blas, L. Seidel, and A. Serrano-Fernández, Enhancing Force Concept Inventory diagnostics to identify dominant misconceptions in first-year engineering physics, Eur. J. Eng. Educ. 35, 597 (2010).
  30. A. Savinainen and P. Scott, The Force Concept Inventory: A tool for monitoring student learning, Phys. Educ. 37, 45 (2002).
  31. A. Savinainen and P. Scott, Using the Force Concept Inventory to monitor student learning and to plan teaching, Phys. Educ. 37, 53 (2002).
  32. A. Savinainen and J. Viiri, The Force Concept Inventory as a measure of students’ conceptual coherence, Int. J. Sci. Math. Educ. 6, 719 (2008).
  33. J. I. Yasuda and M. A. Taniguchi, Validating two questions in the Force Concept Inventory with subquestions, Phys. Rev. ST Phys. Educ. Res. 9, 010113 (2013).
  34. M. Planinić, L. Ivanjek, and A. Sušac, Rasch model based analysis of the Force Concept Inventory, Phys. Rev. ST Phys. Educ. Res. 6, 010103 (2010).
  35. T. F. Scott and D. Schumayer, Students’ proficiency scores within multitrait item response theory, Phys. Rev. ST Phys. Educ. Res. 11, 020134 (2015).
  36. J. Stewart, C. Zabriskie, S. DeVore, and G. Stewart, Multidimensional item response theory and the Force Concept Inventory, Phys. Rev. Phys. Educ. Res. 14, 010137 (2018).
  37. G. W. Fulmer, Validating proposed learning progressions on force and motion using the Force Concept Inventory: Findings from Singapore secondary schools, Int. J. Sci. Math. Educ. 13, 1235 (2015).
  38. L. Bradshaw and J. Templin, Combining item response theory and diagnostic classification models: A psychometric model for scaling ability and diagnosing misconceptions, Psychometrika 79, 403 (2014).
  39. M. A. Norris, Rethinking the Force Concept Inventory: Developing a cognitive diagnostic assessment to measure misconceptions in Newton’s laws, Ph.D. thesis, Virginia Polytechnic Institute and State University, 2021.
  40. K. Wancham, K. Tangdhanakanond, and S. Kanjanawasee, Sex and grade issues in influencing misconceptions about force and laws of motion: An application of cognitively diagnostic assessment, Int. J. Instr. 16, 437 (2023), https://e-iji.net/ats/index.php/pub/article/view/150.
  41. V. Le, J. M. Nissen, X. Tang, Y. Zhang, A. Mehrabi, J. W. Morphew, H.-H. Chang, and B. Van Dusen, Applying cognitive diagnostic models to mechanics concept inventories, Phys. Rev. Phys. Educ. Res. 21, 010103 (2025).
  42. G. Xu and S. Zhang, Identifiability of diagnostic classification models, Psychometrika 81, 625 (2016).
  43. J. de la Torre, A cognitive diagnostic model for cognitively based multiple-choice options, Appl. Psychol. Meas. 33, 163 (2009).
  44. M. Elbulok, A cognitively diagnostic modeling approach to diagnosing misconceptions and subskills, Doctoral dissertation, Columbia University, 2021.
  45. S. Mi, J. Ye, Y. Li, and H. Bi, Development and validation of a conceptual survey instrument to evaluate senior high school students’ understanding of electrostatics, Phys. Rev. Phys. Educ. Res. 19, 010114 (2023).
  46. J. X. Yao and C. S. He, Improving Secondary School Physics Teaching Based on Learning Progressions: Kinematics and Force (Beijing Normal University Press, Beijing, 2020) (in Chinese).
  47. E. Kim and S. J. Pak, Students do not overcome conceptual difficulties after solving 1000 traditional problems, Am. J. Phys. 70, 759 (2002).
  48. M. Reiner, J. D. Slotta, M. T. H. Chi, and L. B. Resnick, Naive physics reasoning: A commitment to substance-based conceptions, Cognit. Instr. 18, 1 (2000).
  49. H. Ş. K𝚤z𝚤lc𝚤k, M. Aygün, E. Şahin, N. Önder-Çelikkanl𝚤, O. Türk, T. Taşk𝚤n, and B. Güneş, Possible misconceptions about solid friction, Phys. Rev. Phys. Educ. Res. 17, 023107 (2021).
  50. D. E. Brown, Students’ concept of force: The importance of understanding Newton’s third law, Phys. Educ. 24, 353 (1989).
  51. J. Klammer, An overview of techniques for identifying, acknowledging and overcoming alternate conceptions in physics education, ERIC Document No. ED423121, ERIC, 1998.
  52. S. Gönen, A study on student teachers’ misconceptions, and scientifically acceptable conceptions about mass and gravity, J. Sci. Educ. Technol. 17, 70 (2008).
  53. N. Sari, Murniati, and S. Ilyas, The implementation of problem-based learning modules to decrease misconception on Newton’s law topic, J. Phys. Conf. Ser. 1460, 012137 (2020).
  54. C. Kruger, D. Palacio, and M. Summers, Surveys of English primary teachers’ conceptions of force, energy, and materials, Sci. Educ. 76, 339 (1992).
  55. S. Bayraktar, Misconceptions of Turkish pre-service teachers about force and motion, Int. J. Sci. Math. Educ. 7, 273 (2009).
  56. A. Caramazza, M. McCloskey, and B. Green, Naive beliefs in “sophisticated” subjects: Misconceptions about trajectories of objects, Cognition 9, 117 (1981).
  57. A. E. Prescott and M. C. Mitchelmore, Teaching projectile motion to eliminate misconceptions, in Proceedings of the 29th Conference of the International Group for the Psychology of Mathematics Education (PME, Melbourne, 2005), Vol. 4, pp. 97–104.
  58. T. Neidorf, A. Arora, E. Erberber, Y. Tsokodayi, and T. Mai, Student Misconceptions and Errors in Physics and Mathematics: Exploring Data from TIMSS and TIMSS Advanced (Springer, Cham, 2020).
  59. A. Mutsvangwa, A study of student teachers’ misconceptions on uniform circular motion, J. Phys. Conf. Ser. 1512, 012029 (2020).
  60. J. de la Torre and N. Minchen, Cognitively diagnostic assessments and the cognitive diagnosis model framework, Psicol. Educ. 20, 89 (2014).
  61. T. M. Haladyna, S. M. Downing, and M. C. Rodriguez, A review of multiple-choice item-writing guidelines for classroom assessment, Appl. Meas. Educ. 15, 309 (2002).
  62. See Supplemental Material at http://link.aps.org/supplemental/10.1103/hrx4-gpd2 for the full original Chinese test and its English translation.
  63. L. Akbay and J. de la Torre, Estimation approaches in cognitive diagnosis modeling when attributes are hierarchically structured, Psicothema 32, 122 (2020).
  64. W. Ma and J. de la Torre, gdina: An r package for cognitive diagnosis modeling, J. Stat. Software 93, 1 (2020).
  65. A. C. George, A. Robitzsch, T. Kiefer, J. Groß, and A. Ünlü, The r package CDM for cognitive diagnosis models, J. Stat. Software 74, 1 (2016).
  66. D. Rizopoulos, ltm: An r package for latent variable modeling and item response theory analyses, J. Stat. Software 17, 1 (2006).
  67. W. Revelle, psych (Version 2.5.6) [r package] (Northwestern University, Evanston, IL, 2025).
  68. J. Chen, J. de la Torre, and Z. Zhang, Relative and absolute fit evaluation in cognitive diagnosis modeling, J. Educ. Measure. 50, 123 (2013).
  69. M. Hansen, L. Cai, S. Monroe, and Z. Li, Limited-information goodness-of-fit testing of diagnostic classification item response models, Br. J. Math. Stat. Psychol. 69, 225 (2016).
  70. W. Wang, L. Song, P. Chen, Y. Meng, and S. Ding, Attribute-level and pattern-level classification consistency and accuracy indices for cognitive diagnostic assessment, J. Educ. Meas. 52, 457 (2015).
  71. R. L. Ebel and D. A. Frisbie, Essentials of Educational Measurement, 5th ed. (Prentice Hall, Englewood Cliffs, NJ, 1991).
  72. M. Young, B.-A. Cummings, and C. St-Onge, Ensuring the quality of multiple-choice exams administered to small cohorts: A cautionary tale, Perspect. Med. Educ. 6, 21 (2017).
  73. F. B. Baker, The Basics of Item Response Theory, 2nd ed. (ERIC Clearinghouse on Assessment and Evaluation, College Park, MD, 2001).
  74. J. C. Nunnally and I. H. Bernstein, Psychometric Theory, 3rd ed. (McGraw-Hill, New York, 1994).
  75. M. Tavakol and R. Dennick, Making sense of Cronbach’s alpha, Int. J. Med. Educ. 2, 53 (2011).
  76. W. A. Nicewander, Conditional reliability coefficients for test scores, Psychol. Methods 23, 351 (2018).
  77. B. F. Green, R. D. Bock, L. G. Humphreys, R. L. Linn, and M. D. Reckase, Technical guidelines for assessing computerized adaptive tests, J. Educ. Measure. 21, 347 (1984).
  78. H. Akaike, A new look at the statistical model identification, IEEE Trans. Autom. Control 19, 716 (1974).
  79. K. P. Burnham and D. R. Anderson, Model Selection and Multimodel Inference, 2nd ed. (Springer, New York, 2002).
  80. G. Schwarz, Estimating the dimension of a model, Ann. Stat. 6, 461 (1978).
  81. Y. Liu, W. Tian, and T. Xin, An application of M2 statistic to evaluate the fit of cognitive diagnostic models, J. Educ. Behav. Stat. 41, 3 (2016).
  82. W. Ma, Evaluating the fit of sequential G-DINA model using limited-information measures, Appl. Psychol. Meas. 44, 167 (2020).
  83. R. C. MacCallum, M. W. Browne, and H. M. Sugawara, Power analysis and determination of sample size for covariance structure modeling, Psychol. Methods 1, 130 (1996).
  84. A. A. diSessa, Knowledge in pieces, in Constructivism in the Computer Age, edited by G. Forman and P. B. Pufall (Lawrence Erlbaum Publishers, New Jersey, 1988).
  85. S. Vosniadou, The development of students’ understanding of science, Front. Educ. 4, 32 (2019).
  86. M. T. H. Chi, J. D. Slotta, and N. de Leeuw, From things to processes: A theory of conceptual change for learning science concepts, Learn. Instr. 4, 27 (1994).
  87. G. J. Posner, K. A. Strike, P. W. Hewson, and W. A. Gertzog, Accommodation of a scientific conception: Toward a theory of conceptual change, Sci. Educ. 66(2), 211 (1982).
  88. J. Tuminaro and E. F. Redish, Elements of a cognitive model of physics problem solving: Epistemic games, Phys. Rev. ST Phys. Educ. Res. 3, 020101 (2007).
  89. C. Wang, H.-H. Chang, and J. Douglas, Combining CAT with cognitive diagnosis: A weighted item selection approach, Behav. Res. Methods Instrum. Comput. 44, 95 (2012).
  90. D. F. Treagust, Development and use of diagnostic tests to evaluate students’ misconceptions in science, Int. J. Sci. Educ. 10, 159 (1988).
  91. K. Jielili, C. W. Yang, and J. X. Yao, Research on the application of cognitive diagnosis based on generative artificial intelligence, J. China Examinations 9, 64 (2025).
  92. S. K. Bitew, J. Deleu, C. Develder, and T. Demeester, Distractor generation for multiple-choice questions with predictive prompting and large language models, arXiv:2307.16338.
  93. S. Moore, R. Schmucker, T. Mitchell, and J. Stamper, Automated generation and tagging of knowledge components from multiple-choice questions, in Proceedings of the Eleventh ACM Conference on Learning @ Scale (Association for Computing Machinery, New York, 2024), pp. 122–133.

Outline

Information

Phys. Rev. Phys. Educ. Res. 22, 010133– Published 9 April, 2026
  • Received 26 September 2025
  • Accepted 24 February 2026
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation