- Editors' Suggestion
- Open Access
Missing data on physics exams: Demographic patterns, course-level predictions, and implications for equity
Phys. Rev. Phys. Educ. Res. 22, 020149 – Published 21 October, 2026
DOI: https://doi.org/10.1103/b4q5-dkn5
Abstract
In a previous quantitative retrospective study, we showed that different demographic groups of students leave different numbers of problems blank on physics exams, leading to inequities in course outcomes. In that work, we argued that there were good reasons to treat these blanks as missing data, rather than indicators of a lack of understanding. In this paper, we refine this analysis and show more detailed breakdowns of uncollected test item responses by race and ethnicity and first-generation college student status, coming to the same conclusion: test item responses are uncollected for students with different ethnic and racial backgrounds at different rates, and these patterns are not exclusive to low-performing students. We also correct an error from our previous work, finding here that there is no significant gender difference in uncollected test item responses. Finally, we provide a more robust analysis of course level data illustrating that blanks are a variable controlled at the course level rather than the student level, providing more evidence for the use of a course deficit model (rather than a student deficit model) when examining equity disparities, and also suggesting that there are plausible means for instructors to minimize uncollected test item responses, and therefore reduce or even eliminate the bias associated with this missing data. We provide a couple suggestions for faculty who want to minimize the impacts of blanks.
Physics Subject Headings (PhySH)
Article Text
References (29)
- G. Matters and P. C. Burnett, Multiple-choice versus short-response items: Differences in omit behaviour, Aust. J. Educ. 43, 117 (1999).
- C. Di Chiacchio, S. De Stasio, and C. Fiorilli, Examining how motivation toward science contributes to omitting behaviours in the Italian PISA 2006 sample, Learn. Individ. Differ. 50, 56 (2016).
- C. Paul, D. J. Webb, M. K. Chessey, and J. Lucas, Pondering zeros : Uncovering hidden inequities within a decade of grades, in 2018 PERC Proceedings, edited by A. Traxler, Y. Cao, and S. Wolf (2018).
- S. Cotner, C. J. Ballen, and A. R. Dalby, Can mixed assessment methods make biology classes more equitable?, PLOS ONE 12, e0189610 (2017).
- M. S. DeCaro, K. G. Golway, C. R. Bego, S. N. Derkson, and J. L. Hieb, Active learning increases gateway engineering course outcomes and closes preparation gap for underrepresented students, J. STEM Educ. Res. 8, 616 (2025).
- D. J. Webb and C. A. Paul, Attributing equity gaps to course structure in introductory physics, Phys. Rev. Phys. Educ. Res. 19, 020126 (2023).
- C. A. Paul and D. J. Webb, Examining equity and graduation rates at two institutions using a course deficit model and the collaborative learning through active sense-making in physics curriculum, Phys. Rev. Phys. Educ. Res. 21, 010107 (2025).
- N. T. Young, R. L. Matz, E. F. Bell, and C. Hayward, Reducing the weight of low exam scores may raise average grades but does not appear to impact equity gaps, arXiv:2508.09906.
- N. M. Stephens, S. A. Fryberg, H. R. Markus, C. S. Johnson, and R. Covarrubias, Unseen disadvantage: How American universities’ focus on independence undermines the academic performance of first-generation college students., J. Pers. Soc. Psychol. 102, 1178 (2012).
- C. Stathopoulou and F. Kalabasis, Language and culture in mathematics education: Reflections on observing a Romany class in a Greek school, Educ. Stud. Math. 64, 231 (2006).
- D. J. Webb and C. A. Paul, High-stakes exams inflate a gender gap and contribute to systematic grading errors in one introductory physics series, Phys. Rev. Phys. Educ. Res. 21, 020134 (2025).
- A. B. Simmons and A. F. Heckler, Grades, grade component weighting, and demographic disparities in introductory physics, Phys. Rev. Phys. Educ. Res. 16, 020125 (2020).
- D. J. Webb, Concepts first: A course with improved educational outcomes and parity for underrepresented minority groups, Am. J. Phys. 85, 628 (2017).
- D. J. Webb, C. A. Paul, and M. K. Chessey, Relative impacts of different grade scales on student success in introductory physics, Phys. Rev. Phys. Educ. Res. 16, 020114 (2020).
- C. A. Paul and D. J. Webb, Percent grade scale amplifies racial or ethnic inequities in introductory physics, Phys. Rev. Phys. Educ. Res. 18, 020103 (2022).
- T. R. Guskey, The case against percentage grades, Educ. Leadersh. 71, 68 (2013).
- I. Rodriguez, E. Brewe, V. Sawtelle, and L. H. Kramer, Impact of equity models and statistical measures on interpretations of educational reform, Phys. Rev. ST Phys. Educ. Res. 8, 020103 (2012).
- D. B. Reeves, The case against the zero, Phi Delta Kappan 86, 324 (2004).
- W. Potter, D. Webb, C. Paul, E. West, M. Bowen, B. Weiss, L. Coleman, and C. De Leone, Sixteen years of collaborative learning through active sense-making in physics (CLASP) at UC Davis, Am. J. Phys. 82, 153 (2014).
- C. Paul, W. Potter, and B. Weiss, Grading by response category: A simple method for providing students with meaningful feedback on exams in large courses, Phys. Teach. 52, 485 (2014).
- C. A. Paul, D. J. Webb, M. K. Chessey, and W. H. Potter, Equity of success in CLASP courses at UC Davis, in 2017 Physics Education Research Conference Proceedings (2017), p. 292.
- M. K. Chessey, Transfer Students Redefining Physics Culture: Student Agency and Responses to Traditional Physics Education, Ph.D. thesis, University of Calif., Davis, 2018, copyright—Database copyright ProQuest LLC; ProQuest does not claim copyright in the individual underlying works; Last updated—2023-03-03.
- J. C. Speirs, W. B. Lane, and N. Laird, Thematic analysis of students’ perceptions of grading practices in physics, Phys. Rev. Phys. Educ. Res. 22, 010115 (2026).
- E. L. Hackerson, T. Slominski, N. Johnson, J. B. Buncher, S. Ismael, L. Singelmann, A. Leontyev, A. G. Knopps, A. McDarby, J. J. Nguyen, D. L. J. Condry, J. M. Nyachwaya, K. T. Wissman, W. Falkner, K. Grieger, L. Montplaisir, A. Hodgson, and J. L. Momsen, Alternative grading practices in undergraduate STEM education: A scoping review, Discip. Interdiscip. Sci. Educ. Res. 6, 15 (2024).
- I. D. Beatty, Standards-based grading in introductory university physics, J. Scholarsh. Teach. Learn. 13, 1 (2013).
- P. Leslie and E. Lundblom, Specifications grading: What it is, and lessons learned, Semin. Speech Lang. 41, 298 (2020).
- R. Armacost and J. Pet-Armacost, Using mastery-based grading to facilitate learning, in 33rd Annual Frontiers in Education, 2003. FIE 2003 (IEEE, Westminster, Colorado, USA, 2003), Vol. 1, pp. T3A_20–T3A_25.
- T. Carey and J. Carifio, The minimum grading controversy: Results of a quantitative study of seven years of grading data from an urban high school, Educ. Res. 41, 201 (2012).
- M. Townsley and D. Schmid, Alternative grading practices: An entry point for faculty in competency-based education, J. Competency-Based Educ. 5, 1 (2020).
