Accuracy on assessments is commonly studied in education research but response time (RT) is relatively less investigated even though decades of research in cognitive sciences indicate that time can be an important dimension for understanding student learning. To better understand RT and the potentially important relations between accuracy and RT in physics education, we conducted an exploratory investigation by collecting and analyzing both accuracy and RT data on physics-relevant math skills on low-stakes pre and posttests as well as course exam scores in algebra-based and calculus-based introductory physics courses over two semesters for a total of $N=1936$ participants. Overall, we found a high level of variation in response times revealing weak but consistent patterns of associations between RT and accuracy on skills and exam scores. First, we found a nonlinear relationship between RT and accuracy on the pretest and on the post-test, which may indicate a variety of strategies and engagement among students on these participation-credit-only tests. Second, the results indicate that while RT alone does not predict course grade, when controlling for accuracy on pre or posttest math skills, students with lower RT on these skills are more likely to get better grades. Therefore, both pre or posttest accuracy and speed predicted course grades, though accuracy explained a substantial amount of variance ($\sim 35\%$) while pretest RT explained a much smaller amount of variance ($\sim 1\%$). Third, controlling for both pretest accuracy and pretest RT, we found that students who sped up from pre to posttest were likely to get higher exam scores; however, students who slowed down were on average likely to have a higher post-test score. Fourth, since systemic inequities in STEM education have been documented via measured mean differences between some demographic groups for exam scores and accuracy on math skills, we compared RTs by sex, race, first-generation status, and citizenship to potentially gain more insight into these inequities. We found no consistent or conclusive evidence of demographic differences, though in multiple comparisons, Black, Hispanic, Native American, and Pacific Islander students had larger RTs on average, and in one comparison they were slightly faster. We found that RT was not a mediator of demographic differences in physics grades, though, as expected, accuracy on math skills was a mediator. We briefly discuss how our results relate to various cognitive models such as cognitive ability, speed-accuracy trade-offs, fluency and cognitive load, dual-process theories, and student psychological factors like self-efficacy, anxiety, and motivation. We argue that, based on which (if any) of the above mechanisms are at play, valuing speed in physics may have benefits, such as improving fluency to reduce cognitive load and drawbacks, such as unintentionally using speed as a proxy for achievement or inducing excessive stress that may interfere with performance and student well-being.

Following the global COVID-19 pandemic, digital learning environments have gained in importance in education all around the world. A new digital learning format called a “Digital Learning Module” (DLM) was developed within the Physics Education Research (PER) group at CERN, consisting of a series of prerecorded explanatory and experiment videos, interactive elements, an interactive simulation experiment, and expert interviews. So far, one DLM has been created on the topic of positron emission tomography (PET). Aiming to establish a set of effective design principles that can be employed in the design of future DLMs or similar digital learning environments, the goal of this research is to evaluate the already existing PET DLM. In doing so, this research brings together different theoretical and empirical studies, in order to identify design principles fitting the format of the DLM. It also reviews key physics literature on PET to identify its disciplinary-relevant aspects. First, an analysis of the PET DLM was performed in terms of the identified design principles and disciplinary-relevant aspects. Then, nine semistructured interviews were conducted with high-school students from Greece and Austria as they worked through the PET DLM. A qualitative content analysis of the interview transcripts has determined the set of effective design principles that will constitute the foundation of future DLMs developed by the CERN PER group and can serve as a starting point and reference for researchers developing similar digital learning environments elsewhere.

Inequity is persistent in science, technology, engineering, and mathematics classrooms despite significant attention to upending it. In order to better understand how and why inequity persists, an analysis is presented of 27 self-identified white male physics faculty and graduate students responding to an interview prompt about the social exclusion of a Black woman by peers in a physics classroom. Three main research questions are addressed: (i) How familiar are these men with the alienation Black women frequently experience?, (ii) To what do they attribute this alienation?, and (iii) What do they think an instructor should do in a classroom where this alienation is occurring? Interviewees were found to have little understanding of the bias and discrimination Black women face, were hesitant to name the underlying cause as racism or sexism, and generally had limited insight into how to address racial and gender bias in their own classrooms. These findings indicate majority group instructors are ill equipped to recognize or disrupt oppressive structures in their own classrooms and may inadvertently exacerbate hostile interactions.

Dual-process theories of reasoning suggest that humans reason using two processes often referred to as process 1 (heuristic) and process 2 (analytic). When presented with a situation requiring any sort of reasoning or decision making, process 1 automatically engages and generates an initial mental model to address the situation. Process 2 may or may not be engaged to assess the initial model as a plausible solution. In a study by Kryjevskaia et al., a “screening” question regarding a pulse on a spring aimed to identify students with relevant content knowledge who nevertheless seemed to rely on process 1 when answering a subsequent “target” question. The study was offered as evidence that dual-process theories can explain some discrepancies in student responses to related questions. The study described here assesses the same pair of questions for their ability to distinguish between incorrect answers that stem from inadequate conceptual understanding and those that stem from reasoning approaches. We use Frederick’s cognitive reflection test as part of this analysis. Our results largely support a dual-process-theories perspective of student reasoning.

Although most teachers recognize the importance of taking investigative, open-ended approaches to students’ learning experiences, implementing them in high school classes can be challenging for teachers. In this work, we analyzed data from multiple sources from a teaching Community of Practice (CoP) to investigate (a) barriers to taking an open-ended approach in teaching labs in physics classes, (b) shifts in teachers’ beliefs about taking an open-ended approach during their engagement in a physics teaching CoP in a partnership program, and (c) a case study of one teacher whose shifts in perceptions about taking an open-ended approach in teaching labs led to her successful implementation in her class. The findings confirm the existence of well-known psychological and structural barriers that can prevent teachers from adopting investigative approaches in teaching physics labs. Moreover, we learned how the interaction of these barriers further complicates the adoption of open-ended approaches in physics classes. The study also revealed a significant gap between teachers’ current practices and their desired methods of conducting labs, particularly in terms of structured versus open-ended approaches. The case study offered deeper insights into how shifts in teaching practices occur through changes in perceptions within a supportive CoP.

[This paper is part of the Focused Collection in Investigating and Improving Quantum Education through Research.] The modern physics course is a crucial gateway for physics majors as it provides an introduction to concepts beyond the scope of the K-12 education. This study collected 167 modern physics syllabi from 127 U.S. research-intensive institutions using publicly available syllabi (51.5%) and private correspondence (48.5%). Due to the size of the corpus being large enough for natural language processing (NLP) to provide significant insight into the curricular content of a course, but still being small enough for human analysis to remain feasible, both NLP and human-analysis methods were utilized. The use of both methods provided a way to test the newly introduced method of NLP in physics education research and offered a novel tool for cross-validation. Public course catalogs were consulted to identify prerequisites and corequisites with 89.3% of students having completed a course in calculus II or higher. Foundational topics like Newtonian mechanics (94%), electricity and magnetism (84.4%), and waves or optics (77.2%) were frequently required. Most institutions consistently taught quantum physics (94%), atomic physics (83%), and relativity (70%), but there was variability in the inclusion of other topics alongside these three main topics. The study highlights the current modern physics curricula at U.S. research-intensive institutions, emphasizing the importance of a consistent and comprehensive education for physics majors across universities. This insight contributes to the ongoing discourse on optimizing physics education in higher education.

There is a growing need for more effective interdisciplinary science instruction across undergraduate degree programs. In addition to supporting students’ connections between disciplinary concepts, interdisciplinary learning can develop students’ critical thinking skills and allow them to evaluate scientific investigations and claims between diverse topics. Physics Education Research literature has particularly focused on introductory physics courses for life sciences students, in part because students majoring in life sciences represent one of the largest demographics enrolled in physics courses. This literature has primarily focused on students’ development of conceptual understanding, modeling skills, and perspectives of the two fields. In this study, we explored how biology and physics undergraduates approach and perceive critical thinking between the two disciplines. We conducted structured think-aloud interviews with biology and physics students, asking students to first complete portions of established biology and physics critical thinking assessments and then respond to several follow-up questions about critical thinking more generally. Using thematic analysis to inductively code interview responses into emergent themes, we found that most students, regardless of major, described different approaches to evaluating biology and physics experiments. However, physics students provided similar definitions of critical thinking in the two disciplines, while biology students provided similar and different definitions in almost equal numbers. The exception was related to the use of quantitative methods solely being associated with critical thinking in physics, despite both critical thinking assessments involving quantitative data analysis. When looking across constructs, we saw no clear trends or relationships between individual students’ responses to each of the interview questions. We also explored students’ broader perspectives on the two fields and found that physics students assume that physics is needed to understand biology but not vice versa, which did not align with their perspectives on critical thinking between disciplines. We use this complexity to motivate future work to understand the impact of biology and physics instruction, as well as other STEM disciplines, on developing students’ critical thinking skills and perceptions.

Research-based assessments (RBAs) allow researchers and practitioners to compare student performance across different contexts and institutions. In recent years, research attention has focused on the student populations these RBAs were initially developed with because much of that research was done with “samples of convenience” that were predominantly white men. Prior research has found that the Force Concept Inventory (FCI) behaved differently for men and women using differential item functioning (DIF) analysis. We extend this research in two ways. First, we test the FCI for DIF across the intersection of gender and race for Asian, Black, Hispanic, White, and White Hispanic men and women. Second, we apply the Eaton and Willoughby five-factor model of the FCI to interpret the results of the DIF analysis. We found large DIF on a large number of FCI items. The patterns of items with large DIF follows the five-factor model. The alignment of DIF with this factor structure, along with the measurement invariance of this factor structure across these ten social identities, indicates that the items on the FCI are likely not biased but are instead measuring real differences in physics knowledge among these groups. We frame these differences as educational debts that society owes to these marginalized groups that physics instruction needs to actively repay.

This study explores the use of artificial intelligence in grading high-stakes physics exams, emphasizing the application of psychometric methods, particularly item response theory, to evaluate the reliability of AI-assisted grading. We examine how grading rubrics can be iteratively refined and how threshold parameters can determine when AI-generated grades are reliable versus when human intervention is necessary. By adjusting thresholds for correctness measures and uncertainty, AI can grade with high precision, significantly reducing grading workloads while maintaining accuracy. Our findings show that AI can achieve a coefficient of determination of $R^2\approx 0.91$ when handling half of the grading load, and $R^2\approx 0.96$ for one-fifth of the load. These results demonstrate AI’s potential to assist in grading large-scale assessments, reducing both human effort and associated costs. However, the study underscores the importance of human oversight in cases of uncertainty or complex problem solving, ensuring the integrity of the grading process.

This article reports on a controlled study to investigate the efficacy of incorporating explicit discussions about the duality of human reasoning and its implications for learning in physics instruction. The central goal is to probe to what extent, if at all, such discussions improve student performance on tasks that tend to elicit intuitively appealing but incorrect responses. We hypothesized that explicitly discussing the reasoning pathways and hazards associated with the interactions between automatic process 1 and deliberate process 2 could boost students’ performance on a variety of topics in physics. In addition, we explored the relationships between student performance in physics, as measured by the Force and Motion Conceptual Evaluation (FMCE), and student tendency toward cognitive reflection, as measured by the cognitive reflection test (CRT). The results revealed a strong link between postinstruction performance on FMCE and the CRT score. However, no measurable effect of explicit discussions on the duality of reasoning on FMCE performance was observed. Possible explanations for the null result and future directions for research are explored.

Previous research has shown that in prehigher education stages, inquiry-based teaching is not sufficient for forming a mature understanding of the nature of science (NOS). However, there is relatively little research conducted on colleges. Inquiry-based teaching should not overlook cognitive frameworks, as students’ limited scientific reasoning abilities may become overwhelmed by the challenges of scientific inquiry. In the college physics laboratory, we manipulated underlying pedagogy (lecture, implicit inquiry, and explicit inquiry) and group compositions (homogeneous and heterogeneous), resulting in a $2\times 3$ teaching treatments randomly across six classes ($n=208$ students) to explore the effectiveness of different teaching methods and grouping strategies in enhancing college students’ understanding of the NOS, as well as the potential relationship between students’ views of the NOS and their scientific reasoning skills. This study found that students’ understanding of the NOS was influenced by the teaching mode, and the teaching mode had different effects on students with varying abilities in scientific reasoning. Explicit inquiry-based teaching was particularly important for enhancing the understanding of the NOS among students with low to medium scientific reasoning abilities.

As quantum technologies transition from the research laboratory into commercial development, the opportunities for students to begin their careers in this new quantum industry are increasing. With these new career pathways, more and more people are considering the best ways to educate students about quantum concepts and relevant skills. In particular, the quantum industry is looking for new employees with experimental skills, but the instructional labs, capstone projects, research experiences, and internships that provide experiences where students can learn these skills are often resource intensive and not available at all institutions. The quantum company, Infleqtion, recently made its online quantum matter machine, Oqtant, publicly available, so people around the world could send commands to create and manipulate Bose-Einstein condensates and receive back real experimental data. Making a complex quantum experiment accessible to anyone has the potential to extend the opportunity to work with quantum experiments to students at less-resourced institutions. As a first step in understanding the potential benefits of using such a platform in educational settings, we collected data from instructors and students who were interested in using, or had used, Oqtant. In this study, we investigate instructors’ views about reasons they would like to use Oqtant and the challenges they would face in doing so. We also provide a concrete example of how Oqtant was used in an upper-division undergraduate quantum mechanics course and the instructor’s perception of its benefits. We complement this with the student perspective, discussing student experiences interacting with Oqtant in their course or through think-aloud interviews outside of a course. This allows us to investigate the reasons students perceive Oqtant to be a real experiment even though they never physically interact with it, how Oqtant compares to their other experimental experiences, and what they enjoy about working with it. These results will help the community consider the potential value of creating more opportunities for students to access remote quantum experiments.

When learning physics, students need more than just an understanding of mathematical and physical concepts. Integrating the two fields is crucial, as research indicates that students often struggle even when they have a strong grasp of both. In this paper, we use the heat equation as an example from higher education. Given the importance of the gradient and the Laplacian in the heat equation, as well as fluency and flexibility in using multiple external representations, this study investigates students’ reasoning about the gradient and the Laplacian of a function within different function representations in mathematics and physics contexts. To do so, we designed isomorphic open-response questions, administered as paper-and-pencil tests to 190 first-year students who were taking calculus-based introductory physics courses and multivariable calculus courses during their second semester. These students were enrolled in physics, mathematics, engineering, and bioengineering programs. We found that students face conceptual challenges with both the gradient and the Laplacian. Student difficulties with interpreting the gradient may be related to an incomplete understanding of steepness in graphical representations of functions of two independent variables. Moreover, their understanding of the Laplacian is limited. While context did not significantly affect students’ reasoning in terms of the types of difficulties they encountered, the proportion of students facing these difficulties, and their tendency to link or discuss more than one aspect of a concept, we did find context-specific prevalence of graphical, algebraic-symbolic, or linguistic descriptions. Students used linguistic descriptions more frequently in a physics context. We observed varying abilities to express the meaning of concepts in different answer representations. Linguistic descriptions of concepts proved particularly challenging for both concepts and contexts. Our findings provide insight into students’ understanding of two concepts related to the 2D heat equation.

This study investigated how students reason about the partial derivative and the directional derivative of a multivariable function at a given point, using different graphical representations for the function in the problem statement. Questions were formulated to be as isomorphic as possible in both mathematics and physics contexts and were given as pen-and-paper tests to 190 mathematics, physics, engineering, and bioengineering students in their second semester. Our first research question asked how students reason about partial derivatives and directional derivatives with different function representations and in different contexts. Our second research question investigated whether a physics context hinders or improves students’ understanding of these concepts. Performance on partial derivative questions varied with both the representations students used in their answers and the function representations used in the question. Students performed better when they provided algebraic-symbolic answers and when they answered Cartesian graph questions. For directional derivatives, students struggled with correct interpretations, often misinterpreting steepness as average steepness between points rather than as steepness at a specific point. The context did not significantly affect overall performance, but it did influence the representations students used in their answers, with a tendency toward linguistic descriptions in physics.

We used student surveys to assess changes in proficiency with integral calculus, calculus self-efficacy, and physics self-efficacy in a second semester course of introductory physics focused on electricity and magnetism. In addition to tracking the fraction of quantitative problems solved correctly on the calculus proficiency survey, we also used principal component analysis and factor analysis of the data collected using all three surveys to evaluate changes in student proficiency and attitude. These analyses indicate that while all students demonstrated an increase in proficiency with integral calculus, including using integration in the context of physics, women displayed larger gains than men. Conversely, men showed larger gains in calculus self-efficacy. When combined, these data suggest that student identity modulates the correlation between a student’s calculus abilities and their perception or self-evaluation of those abilities. We further show that calculus self-efficacy and physics self-efficacy are more strongly correlated for women than for men. These data suggest that student identity affects the coupling between calculus self-efficacy and physics self-efficacy, which could be consistent with previously published work demonstrating that mathematics self-efficacy is modulated by student identity [A. C. Frenzel et al., Learn. Instr. 17, 478 (2007); A. C. Frenzel et al., Eur. J. Psychol. Educ. 22, 497 (2007); R. A. Louis and J. M. Mistele, Int. J. Sci. Math. Educ. 10, 1163 (2012)]. These results may thus also highlight a potential contributing factor to gender-related differences in physics self-efficacy [J. M. Nissen and J. T. Shemwell, Phys. Rev. Phys. Educ. Res. 12, 020105 (2016); E. M. Marshman et al., Phys. Rev. Phys. Educ. Res. 14, 020123 (2018); T. Espinosa et al., Phys. Rev. Phys. Educ. Res. 15, 010132 (2019)] as well as the complexity of addressing those differences.

As quantum technologies transition out of the research lab and into commercial applications, it becomes important to better prepare students to enter this new and evolving workforce. To work toward this goal of preparing physics students for a career in the quantum industry, a senior capstone course called “Quantum Forge” was created at the University of Colorado Boulder. This course aims to provide students with a hands-on quantum experience and prepare them to enter the quantum workforce directly after their undergraduate studies. Some of the course’s goals are to have students understand what comprises the quantum industry and have them feel confident they could enter the industry if desired. To understand to what extent these goals are achieved, we followed the first cohort of Quantum Forge students through their year in the course in order to understand their perceptions of the quantum industry, including what it is, whether they feel that they could be successful in it, and whether or not they want to participate in it. The results of this work can assist educators in optimizing the design of future quantum-industry-focused courses and programs to better prepare students to be a part of this burgeoning industry.

Recent advancements in large language models (LLMs) hold significant promise for improving physics education research that uses machine learning. In this study, we compare the application of various models for conducting a large-scale analysis of written text grounded in a physics education research classification problem: identifying skills in students’ typed lab notes through sentence-level labeling. Specifically, we use training data to fine-tune two different LLMs, BERT and LLaMA, and compare the performance of these models to both a traditional bag-of-words approach and a few-shot LLM (without fine-tuning). We evaluate the models based on their resource use, performance metrics, and research outcomes when identifying skills in lab notes. We find that higher-resource models often, but not necessarily, perform better than lower-resource models. We also find that all models report similar trends in research outcomes, although the absolute values of the estimated measurements are not always within uncertainties of each other. We use the results to discuss relevant considerations for education researchers seeking to select a model type for use as a classifier.

We use a validated conceptual multiple-choice survey instrument focusing on thermodynamic processes and the first and second laws of thermodynamics at the level of introductory physics to investigate the problem-property dependence of introductory and advanced student responses to introductory thermodynamics problems after traditional lecture-based instruction. The survey instrument has qualitative problems involving the same concepts across multiple problems related to internal energy, work, heat transfer, and entropy. The concepts for which we investigated problem-property dependence include, among others, (i) internal energy and entropy are state variables while work and heat transfer are path-dependent variables, (ii) internal energy is proportional to the absolute temperature for an ideal gas, (iii) work corresponds to the (signed) area under the curve on a PV (pressure-volume) diagram, and (iv) internal energy is constant but entropy increases in isolated systems undergoing spontaneous and irreversible processes. This study used survey data from over 1000 college students in introductory-level algebra-based and calculus-based physics courses as well as upper-level thermodynamics courses; the survey was administered after traditional instruction in relevant concepts for each group. Think-aloud interviews were carried out to gain additional insight into students’ thinking as they responded to the survey problems. For concepts related to internal energy, heat transfer, and work, student responses for different concepts investigated often showed strong problem-property dependence, but advanced students, as a group, generally performed better than introductory students across different problems. For entropy concepts, introductory students consistently performed poorly across problem types, reflecting a persistent belief in the constancy of entropy. By contrast, upper-level students struggled consistently in cases where entropy was not increasing (e.g., net entropy change in a cyclic process). Our systematic investigation of problem-property dependence of student responses is novel for thermodynamics, made possible by the use of a survey that includes multiple varied problem scenarios involving the same underlying concept. In addition to yielding many previously unreported results regarding problem-property dependence in thermodynamics, our work confirms and extends, to new problem settings and student groups, findings previously reported through the study of more limited problem settings and student groups.

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] This study examines the feasibility and potential advantages of using large language models, in particular GPT-4o, to perform partial credit grading of large numbers of student written responses to introductory level physics problems. Students were instructed to write down verbal explanations of their reasoning process when solving one conceptual and two numerical calculation problems on two exams. The explanations were then graded according to a three-item rubric with each item graded as binary (1 or 0). We first demonstrate that machine grading using GPT-4o with no examples or reference answers can reliably agree with human graders in 70%–80% of all cases, which is equal to or higher than the level at which two human graders agree with each other. Two methods are essential for achieving this level of accuracy: (i) Adding explanation language to each rubric item that targets the errors of initial machine grading. (ii) Running the grading process 5 times and taking the most frequent outcome. Next, we show that the variation in outcomes across five machine grading attempts can serve as a grading confidence index. The index allows a human expert to identify $\sim 40\%$ of all potentially incorrect gradings by reviewing just 10%–15% of all responses with the highest variation. Finally, we show that it is straightforward to use GPT-4o to write a clear and detailed explanation of the partial credit grading outcome. Those explanations can be used as feedback for students, which will allow students to understand their grades and raise different opinions when necessary. Almost all feedback messages generated were rated three or above on a five-point scale by two instructors who had taught the course multiple times. The entire grading and feedback generating process costs roughly $5 per 100 student answers, which shows immense promise for automating labor-intensive grading process through a combination of machine grading with human input and supervision.