- Physical Review Journals
All JournalsPhysics Magazine

Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access

Incentivizing supplemental math assignments and using AI-generated hints is associated with improved exam performance

Yifan Lu1,*, K. Supriya2,†, Shanna Shaked2,3, Elizabeth H. Simmons4, and Alexander Kusenko1,5

  • *Contact author: yifanlu@ucla.edu
  • †Contact author: ksupriya@ceils.ucla.edu
PDF

Phys. Rev. Phys. Educ. Res. 21, 010160 – Published 12 June, 2025

DOI: https://doi.org/10.1103/v93w-ytyc

Abstract

Inequities in student access to trigonometry and calculus are often associated with racial and socioeconomic privilege, and often influence introductory physics course performance. To mitigate these disparities in student preparedness, we developed a two-pronged intervention consisting of (1) incentivized supplemental math assignments and (2) AI-generated learning support tools in the form of optional hints embedded in the physics homework assignments. Both interventions are grounded in the situated expectancy-value theory of achievement motivation, which posits that students are more likely to complete a task that they expect to do well in and whose outcomes they think are valuable. For the supplemental math assignments, the extra credit was scaled to make it worth more points for students with lower exam scores, thereby creating even greater value for students who might benefit most from the assignments. AI-generated hints were integrated into the homework assignments, thereby reducing or eliminating the cost to the student, in terms of time, energy, and social barriers or fear of judgment. Our findings indicate that both these interventions are associated with increased exam scores; in particular, the scaled extra credit reduced disparities in completion of supplemental math assignments. These interventions, which are relatively simple for any instructor to implement, are therefore very promising for creating more equitable undergraduate quantitative courses.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (52)

  1. H. Hudson and W. McIntire, Correlation between mathematical skills and success in physics, Am. J. Phys. 45, 470 (1977).
  2. P. M. Sadler and R. H. Tai, Success in introductory college physics: The role of high school preparation, Sci. Educ. 85, 111 (2001).
  3. D. E. Meltzer, The relationship between mathematics preparation and conceptual learning gains in physics: A possible “hidden variable” in diagnostic pretest scores, Am. J. Phys. 70, 1259 (2002).
  4. R. L. Forrest, D. W. Stokes, A. B. Burridge, and C. D. Voight, Math remediation intervention for student success in the algebra-based introductory physics course, Phys. Rev. Phys. Educ. Res. 13, 020137 (2017).
  5. D. Battey, Access to mathematics: “A possessive investment in whiteness”1, Curric. Inq. 43, 332 (2013).
  6. D. M. Bressoud, The strange role of calculus in the United States, ZDM–Math. Educ. 53, 521 (2021).
  7. C. J. Ackins, Equity in opportunity: Minority student enrollment in advanced, advanced placement, and international baccalaureate English and Mathematics courses in Virginia Public High Schools, Ed.D. dissertation, Virginia Tech, 2022, http://hdl.handle.net/10919/107829,
  8. J. Briones, Racialized tracking: The widening of the calculus achievement gap for black and latinx students, Ronald E. McNair Scholars Res. J. 15, 14 (2023).
  9. A. R. Golden, E. N. Srisarajivakul, A. J. Hasselle, R. A. Pfund, and J. Knox, What was a gap is now a chasm: Remote schooling, the digital divide, and educational inequities resulting from the COVID-19 pandemic, Curr. Opin. Psychol. 52, 101632 (2023).
  10. M. Schultz. Impact of the COVID-19 pandemic on achievement gaps in high school: A quantitative analysis of SAT math scores, Ph.D. thesis, Indiana Wesleyan University, 2024, https://www.proquest.com/docview/3064925036?pq-origsite=gscholar&fromopenview=true&sourcetype=Dissertations%20&%20Theses.
  11. S. K. Haderlein, A. R. Saavedra, M. S. Polikoff, D. Silver, A. Rapaport, and M. Garland, Disparities in educational access in the time of COVID: Evidence from a nationally representative panel of American Families, AERA Open 7, 23328584211041350 (2021).
  12. G. M. Francom, S. J. Lee, and H. Pinkney, Technologies, challenges and needs of K-12 teachers in the transition to distance learning during the COVID-19 pandemic, TechTrends 65, 589 (2021).
  13. P. Ong, Covid-19 and the digital divide in virtual learning (2020), https://escholarship.org/uc/item/07g5r002.
  14. E. A. Vogels, A. Perrin, L. Rainie, and M. Anderson, 53% of americans say the internet has been essential during the covid-19 outbreak: Americans with lower incomes are particularly likely to have concerns related to the digital divide and the digital” homework gap”, Pew Research Center, 30 (2020).
  15. V. B. Meling, M.-A. Mundy, L. Kupczynski, and M. E. Green, Supplemental instruction and academic success and retention in science courses at a hispanic-serving institution, World J. Educ. 3, 11 (2013), https://eric.ed.gov/?id=EJ1158634.
  16. L. A. DeSilva, A. Pullen, and J. E. Hasbun, Enhancing student performance in introductory physics in topics related to electricity and magnetism through the use of voluntary workshops, Eur. J. Phys. 39, 035702 (2018).
  17. E. Burkholder, S. Salehi, and C. E. Wieman, Mixed results from a multiple regression analysis of supplemental instruction courses in introductory physics, PLoS One 16, e0249086 (2021).
  18. B. D. Mikula and A. F. Heckler, Framework and implementation for improving physics essential skills via computer-based practice: Vector math, Phys. Rev. Phys. Educ. Res. 13, 010122 (2017).
  19. S. DeVore, E. Marshman, and C. Singh, Challenge of engaging all students via self-paced interactive electronic learning tutorials for introductory physics, Phys. Rev. Phys. Educ. Res. 13, 010127 (2017).
  20. M. S. Hardy, Extra credit: Gifts for the gifted? Teach. Psychol. 29, 233 (2002).
  21. M. A. Harrison, D. G. Meister, and A. J. LeFevre, Which students complete extra-credit work, Coll. Stud. J. 45, 550 (2011).
  22. K. Supriya, C. Bang, J. Ebie, C. Pagliarulo, D. Tucker, K. Villegas, C. Wright, and S. Brownell, Optional exam retakes reduce anxiety but may exacerbate score disparities between students with different social identities, CBE–Life Sci. Educ. 23, ar30 (2024).
  23. R. Singh, M. Saleem, P. Pradhan, C. Heffernan, N. T. Heffernan, L. Razzaq, M. D. Dailey, C. O’Connor, and C. Mulcahy, Feedback during web-based homework: The role of hints, in Artificial Intelligence in Education, edited by G. Biswas, S. Bull, J. Kay, and A. Mitrovic (Springer, Berlin, Heidelberg, 2011), pp. 328–336, 10.1007/978-3-642-21869-9_43.
  24. M. Feng, N. Heffernan, K. Collins, C. Heffernan, and R. F. Murphy, Implementing and evaluating ASSISTments online math homework support at large scale over two years: Findings and lessons learned, in Artificial Intelligence in Education, edited by N. Wang, G. Rebolledo-Mendez, N. Matsuda, O. C. Santos, and V. Dimitrova (Springer Nature Switzerland, Cham, 2023), pp. 28–40, 10.1007/978-3-031-36272-9_3.
  25. K. Sale and K. Muldner, Learning with an algebra computer tutor: What type of hint is best?, in Proceedings of the Annual Meeting of the Cognitive Science Society (2019), Vol. 41, https://escholarship.org/uc/item/8gv8k0t5.
  26. C. Cody, M. Maniktala, N. Lytle, M. Chi, and T. Barnes, The impact of looking further ahead: A comparison of two data-driven unsolicited hint types on performance in an intelligent data-driven logic tutor, Int. J. Artif. Intell. Educ. 32, 263 (2022).
  27. B. G. Johnson, F. Phillips, and L. G. Chase, An intelligent tutoring system for the accounting cycle: Enhancing textbook homework with artificial intelligence, J. Account. Educ. 27, 30 (2009).
  28. P. M. Phothilimthana and S. Sridhara, High-coverage hint generation for massive courses: Do automated hints help CS1 students?, in Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE ’17) (Association for Computing Machinery, New York, NY, 2017), pp. 182–187, 10.1145/3059009.3059058.
  29. V. Rus, Large language models and their implications for conversational tutors and gift, in Proceedings of the 12th Annual GIFT Users Symposium, edited by A. M. Sinatra (2024), pp. 77–86.
  30. J. S. Eccles and A. Wigfield, From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation, Contemp. Educ. Psychol. 61, 101859 (2020).
  31. J. Eccles, T. Adler, R. Futterman, S. Goff, C. Kaczala, J. Meece, and C. Midgley, Expectancies, values, and academic behaviors, in Achievement and Achievement Motivation, edited by J. Spence (W.H. Freeman, San Francisco, CA, 1983), pp. 75–146.
  32. J. W. Atkinson, Motivational determinants of risk-taking behavior, Psychol. Rev. 64, 359 (1957).
  33. T. Perez, J. G. Cromley, and A. Kaplan, The role of identity development, values, and costs in college stem retention, J. Educ. Psychol. 106, 315 (2014).
  34. K. A. Robinson, Y.-k. Lee, E. A. Bovee, T. Perez, S. P. Walton, D. Briedis, and L. Linnenbrink-Garcia, Motivation in transition: Development and roles of expectancy, task values, and costs in early college engineering, J. Educ. Psychol. 111, 1081 (2019).
  35. F. Wu, W. Fan, C. Arbona, and D. de la Rosa-Pohl, Self-efficacy and subjective task values in relation to choice, effort, persistence, and continuation in engineering: An expectancy-value theory perspective, Eur. J. Eng. Educ. 45, 151 (2020).
  36. J. Appianing and R. N. Van Eck, Development and validation of the value-expectancy stem assessment scale for students in higher education, Int. J. STEM Educ. 5, 1 (2018).
  37. T. Perez, S. V. Wormington, M. M. Barger, R. D. Schwartz-Bloom, Y.-k. Lee, and L. Linnenbrink-Garcia, Science expectancy, value, and cost profiles and their proximal and distal relations to undergraduate science, technology, engineering, and math persistence, Sci. Educ. 103, 264 (2019).
  38. A. Wigfield and J. Eccles, Expectancy-value theory of achievement motivation, Contemp. Educ. Psychol. 25, 68 (2000).
  39. A. A. Jack, (no) harm in asking: Class, acquired cultural capital, and academic engagement at an elite university, Sociol. Educ. 89, 1 (2016).
  40. A. Yee, The unwritten rules of engagement: Social class differences in undergraduates’ academic strategies, J. Higher Educ. 87, 831 (2016).
  41. S. J. Spencer, C. Logel, and P. G. Davies, Stereotype threat, Annu. Rev. Psychol. 67, 415 (2016).
  42. D. Asai, Excluded, J. Microbiol. Biol. Educ. 21, 10 (2020).
  43. D. Hestenes, M. Wells, and G. Swackhamer et al., Force concept inventory, Phys. Teach. 30, 141 (1992).
  44. J. Epstein, Development and validation of the calculus concept inventory, in Proceedings of the 9th International Conference on Mathematics Education in a Global Community (Citeseer, Charlotte, NC, 2007), Vol. 9, pp. 165–170.
  45. J. G. MacKinnon and H. White, Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties, J. Econom. 29, 305 (1985).
  46. T. pandas development team, pandas-dev/pandas: Pandas (2020), 10.5281/zenodo.13819579.
  47. Wes McKinney, Data structures for statistical computing in Python, in Proceedings of the 9th Python in Science Conference, edited by Stéfanvan der Walt and Jarrod Millman (2010), pp. 56–61, 10.25080/Majora-92bf1922-00a.
  48. J. D. Hunter, Matplotlib: A 2d graphics environment, Comput. Sci. Eng. 9, 90 (2007).
  49. M. L. Waskom, seaborn: Statistical data visualization, J. Open Source Softwaare 6, 3021 (2021).
  50. W. Yeadon and T. Hardy, The impact of AI in physics education: a comprehensive review from GCSE to university levels, Phys. Educ. 59, 025010 (2024).
  51. L. Ding, T. Li, S. Jiang, and A. Gapud, Students’ perceptions of using ChatGPT in a physics class as a virtual tutor, Int. J. Educ. Technol. Higher Educ. 20, 63 (2023).
  52. E. M. Bensimon, The misbegotten URM as a data point, Center for Urban Education, Rossier School of Education, University of Southern California, Los Angeles (2016).

Outline

Information

Phys. Rev. Phys. Educ. Res. 21, 010160– Published 12 June, 2025
  • Received 3 January 2025
  • Accepted 27 May 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation