This study aimed at evaluating how students perceive the linguistic quality and scientific accuracy of ChatGPT responses to physics comprehension questions. A total of 102 first- and second-year physics students were confronted with three questions of progressing difficulty from introductory mechanics (rolling motion, waves, and fluid dynamics). Each question was presented with four different responses. All responses were attributed to ChatGPT, but in reality, one sample solution was created by the researchers. All ChatGPT responses obtained in this study were wrong, imprecise, incomplete, or misleading. We found little differences in the perceived linguistic quality between ChatGPT responses and the sample solution. However, the students rated the overall scientific accuracy of the responses significantly differently, with the sample solution being rated best for the questions of low and medium difficulty. The discrepancy between the sample solution and the ChatGPT responses increased with the level of self-assessed knowledge of the question content. For the question of highest difficulty (fluid dynamics) that was unknown to most students, a ChatGPT response was rated just as good as the sample solution. Thus, this study provides data on the students’ perception of ChatGPT responses and the factors influencing their perception. The results highlight the need for careful evaluation of ChatGPT responses both by instructors and students, particularly regarding scientific accuracy. Therefore, future research could explore the potential of similar “spot the bot” activities in physics education to foster students’ critical thinking skills.

[This paper is part of the Focused Collection on Qualitative Methods in PER: A Critical Examination.] Since the beginning of the 21st century, the appropriation of the sociocultural perspective by the physics education research community has represented a linguistic turn in the field, pointing out a promising path to overcome the dominance of the “individual paradigm,” both in terms of student learning and initial and continuing teacher training. This approach views science, science education, and research as human social activities embedded in larger sociocultural and institutional contexts, implying a significant theoretical importance to the role of social interaction and the context in which these interactions occur, viewing them as critical to a better understanding of the learning process rather than merely as a secondary role. In this theoretical framework, language plays a fundamental role as a mediator of human action, notably, it is the main system of signs used by humankind. Hence, we recognize the origin of sociocultural perspectives in Lev Vygotsky’s sociohistorical psychology. The neo-Vygotskian James Wertsch proposes a “continuity” of Vygotsky’s theory by emphasizing one of its unexplored assumptions: the characterization of human action as an activity mediated by signs and instruments. In this theoretical construction of a sociocultural approach to human action, the philosophy of language of Mikhail Bakhtin Circle becomes crucial. Data of discursive nature (oral and written speech of the instructor and their pupils, textbooks, or official documents) may be analyzed in physics education research, particularly those focusing on classroom situations (typically didactic interventions). We employed Bakhtinian analysis to avoid the text’s objectivist (positivist) stance and the structural deterministic idea of ideological interpellation suggested by Althusser and endorsed by Pêcheux’s discourse analysis. In order to contribute to a qualitative research technique for discursive data analysis, we explain the Circle’s theory and translate it into possible methodologies for research in physics education. We suggest an “analytical trajectory” based on this as a possible arrangement of the interpretation of discursive data under Bakhtin’s metalinguistic. Finally, we exemplify the use of this analytical trajectory in our research group’s works.

Underrepresentation of women in physics is a prominent issue in the western countries. Since physics teachers are in a unique position to affect new generations of students, it has been suggested that they are an important part of the solution. In this paper, we explore how trainee physics teachers create spaces for themselves as learners of physics while negotiating their positioning as women and trainee teachers. The empirical data consist of interviews with 17 trainee physics students, and the analysis focuses predominantly on the identity negotiations of three woman students. We find that the women simultaneously submit to and master a “physics nerd” discourse that connects physics with nerdiness, masculinity, and intelligence, which enables them to successfully create subject positions incorporating physics student, teacher-student, femininity, and constructive study practice. This is of particular importance to trainee physics teachers, who will be responsible for creating inclusive and productive physics learning environments for their students.

Problem Roulette (PR), an online study service at the University of Michigan, offers points-free formative practice to students preparing for examinations in introductory science, technology, engineering, and mathematics courses. Using four years of PR data involving millions of problem attempts by thousands of students, we quantify the benefits of increased practice study volume in introductory physics. After conditioning the mean final grade on standardized (ACT and SAT) math test score, we analyze deviations based on student study volume. We find a strong effect; mean course grade rises quadratically with the logarithm of the total number of PR questions encountered over the term ($N_{\mathrm{Q},\mathrm{tot}}$), with an overall gain of $0.77\pm 0.12$ grade points between $1<N_{\mathrm{Q},\mathrm{tot}}<1000$. The gains are persistent across the range of math test scores represented in our sample. While $N_{\mathrm{Q},\mathrm{tot}}$ surely correlates with other study habits, the benefits of increased study in general still hold. A model for final grade using test score and study volume largely accounts for demographic stratification, including by sex, parental education level, number of parents at home, nationality or underrepresented minority status, and regional income level, with two significant exceptions: students whose parents did not earn a college degree, who earn $-0.27\pm 0.04$ grade points ($6.1\sigma$) below expectations, and underrepresented minority students at $-0.14\pm 0.04$ points ($3.6\sigma$). Residual scatter in final grade remains comparable to the maximal study gains, implying that the model is far from deterministic: individual variation trumps mean trends. Our findings can help motivate students to study more and help teachers to identify which types of students may especially need such encouragement.

Despite the high demand for STEM (science, technology, engineering, and mathematics) professionals, the STEM talent pipeline continues to leak. A framework of STEM identity could provide insight into why and to what extent individuals engage in STEM-related activities. The present study describes the process of developing and validating an instrument for assessing students’ STEM identity. This instrument was created using the conceptual framework of science identity as its foundation. It aims to conceptualize student STEM identity and extend the STEM identity model. This developmental instrument was examined through expert review, small-scale pilot tests, student interviews, and large-scale field tests. The present study involved 2100 Chinese adolescents from middle and high schools in 10 regions. Factor analysis and Rasch analysis provided evidence of reliability and validity. The final instrument contained four dimensions: recognition, belonging, performance and competence, and interest. Our findings provide multiple sets of evidence for reliability and validity, supporting the appropriate structure of the instrument. It exhibits strong psychometric properties, which can be used to assess sustainability in students’ STEM fields.

With the increasing industrial relevance of new quantum technologies, a well-educated quantum workforce becomes increasingly crucial. The foreseeable lack of workforce raises important questions. What are the expectations regarding the future relevance of second-generation quantum technologies? What are the requirements for the workforce in the coming quantum industry? Which competences, knowledge, and skills should future employees have? In this paper, we report the results of our study that were aimed at mapping requirements and forecasts for the future quantum workforce. Our study consisted of three consecutive survey rounds. In total, we gathered 188 responses from industry and academic experts across Europe. Our study results served as an input for the development of the European Competence Framework for Quantum Technologies, delivered by the project QTEdu CSA for the European Quantum Flagship. In addition, we will discuss predictions from experts related to the future quantum workforce, including the expected industrial relevance of the main areas of quantum technologies, the need for educational efforts, and the expected influence of quantum technologies on everyday life.

Multicultural education invites teachers to support students in critiquing the foundations of a given discipline, with the aim of reimagining that discipline and the purposes it serves. In this paper, we present a series of cases in which high school physics teachers who are enrolled in a summer professional development course expressed vexation as they tried to integrate equity with the physics concept of energy and in which one teacher made significant progress in this integration. These cases serve to illustrate what teacher learning about multicultural education might look like in physics and what resources may support this learning. These cases also point us to some of the ways in which physics as a discipline and schooling as a system make it difficult for teachers to critically examine the canon.

This study examines high school preparation measures [ACT/SAT scores, high school grade point average (HSGPA), and conceptual physics pretest scores], in-class behavior measures (homework submission rates and lecture attendance rates), and in-class achievement measures (homework and test averages) for the last two fully face-to-face prepandemic and the first two fully face-to-face postpandemic semesters of an introductory calculus-based electricity and magnetism class. This class was offered at a large eastern land grant university in the United States. The total number of students for the four semesters was 1033. While some significant differences were measured (higher postpandemic HSGPA, lower postpandemic conceptual pretest scores, higher postpandemic homework average (fall semesters only), and lower postpandemic lecture attendance (spring semesters only), none were larger than a small effect. As such, student achievement, attendance rates, and assignment completion rates were largely unchanged after the pandemic.

In the summer of 2020, as COVID-19 limited in-person research opportunities and created additional barriers for many students, institutions either canceled or remotely hosted their research experience for undergraduates (REU) programs. The present qualitative phenomenographic study was designed to explore some of the possible limitations, challenges, and outcomes of this remote experience. Overall, 94 interviews were conducted with paired participants; mentees ($N=10$) and mentors ($N=8$) from six different REU programs. By drawing on cultural-historical activity theory as a framework, our study uncovers some of the challenges mentees faced while pursuing their research objectives and academic goals. These challenges included motivation, limited access to technology at home, limited communication among REU students, barriers in mentor-mentee relationships, and differing expectations about doing research. Despite the challenges, all mentees reported that this experience was highly beneficial. Comparisons between the outcomes of these remote REUs and published outcomes of in-person undergraduate research programs reveal many similar benefits, including student integration into science, technology, engineering, and mathematics culture. Our study suggests that remote research programs could be considered a means to expand access to undergraduate research experiences even after COVID-19 restrictions have been lifted.

Rubric-based admissions are claimed to help make the graduate admissions process more equitable, possibly helping to address the historical and ongoing inequities in the U.S. physics graduate school admissions process that have often excluded applicants from minoritized races, ethnicities, genders, and backgrounds. Yet, no studies have examined whether rubric-based admissions methods represent a fundamental change in the admissions process or simply represent a new tool that achieves the same outcome. To address that, we developed supervised machine learning models of graduate admissions data collected from our department over a seven-year period. During the first four years, our department used a traditional admission process and switched to a rubric-based process for the following three years, allowing us to compare which parts of the applications were driving admissions decisions. We find that faculty focused on applicants’ physics GRE scores and grade-point averages when making admissions decisions before the implementation of the rubric. While we were able to develop a sufficiently good model whose results we could trust for the data before the implementation of the rubric, we were unable to do so for the data collected after the implementation of the rubric, despite multiple modifications to the algorithms and data such as implementing Tomek Links. Our inability to model the second dataset despite being able to model the first combined with model comparison analyses suggests that rubric-based admission does change the underlying process. These results suggest that rubric-based holistic review is a method that could make the graduate admission process in physics more equitable.

Kinetic energy is usually the entry point for the study of energy in physics and is often perceived as unproblematic. We present evidence, however, that some learners who seem to have accepted the concept, from elementary school students to college physics majors and in-service teachers, nevertheless do not consistently attribute kinetic energy to moving objects that are being pushed or carried by other objects. Factors that seem to contribute to this idea include that the passive object is not moving “on its own”; the lack of attributes, like wheels, that suggest the ability to move on its own; and the perception that it would stop immediately if the driving object were to stop or disappear. We interpret these observations in terms of a model of a conceptual change via assimilation rather than accommodation and suggest some possible instructional implications.

Massive pretrained language models have garnered attention and controversy due to their ability to generate humanlike responses: Attention due to their frequent indistinguishability from human-generated phraseology and narratives and controversy due to the fact that their convincingly presented arguments and facts are frequently simply false. Just how humanlike are these responses when it comes to dialogues about physics, in particular about the standard content of introductory physics courses? This case study explores that question by having ChatGPT, the preeminent language model in 2023, work through representative assessment content of an actual calculus-based physics course and grading the responses in the same way human responses would be graded. As it turns out, ChatGPT would narrowly pass this course while exhibiting many of the preconceptions and errors of a beginning learner. A discussion of possible consequences for teaching, testing, and physics education research is provided as a possible starter for more detailed studies and curricular efforts in the future.

We present an analysis of students’ thinking skills as evidenced by free-response exam problems during the Covid-19 pandemic. We compare two inquiry-based, laboratory-based classical mechanics courses, one taught online and one taught in person during the pandemic, and two inquiry-based, laboratory-based electricity and magnetism courses, one taught online and the other in person during the pandemic. We use a rubric that was previously developed based on Bloom’s taxonomy (revised version) to compare the thinking skills of students in classes taught by different pedagogies. We discuss the method and analysis, and present results and interpretations. No significant differences were found in thinking skills between students in the online and in-person pandemic classical mechanics courses. However, we did see a difference in the thinking skills between the online and in-person pandemic electricity and magnetism courses as the semester progressed.

Preparing for high-stakes exams in introductory physics courses is generally a self-regulated activity. Compared to other exam reviewing strategies, doing practice exams has been shown to help students recognize gaps in their knowledge, encourage active practicing, and produce long-term retention. However, many students, particularly students who are struggling with the course material, are not guided by research-based study strategies and do not use practice exams effectively. Using data collected from a fully online course in Spring 2021, this study examines two interventions aimed at improving student self-regulated studying behaviors and enhancing student metacognition during exam preparation. We found that a modified format of online practice exams with one attempt per question and delayed feedback, increases the accuracy of feedback about student readiness for exams but does not change the accuracy of their predicted exam scores or studying behaviors. Additionally, an added mock exam one week before the actual exam impacts students’ intentions for studying but does not impact actual study behaviors or facilitate metacognition. These results suggest that interventions designed to improve exam preparation likely need to include explicit instruction on study strategies and student beliefs about learning.

Electricity and magnetism are closely related phenomena with a well-known symmetry found in Maxwell equations. An essential part of any electricity and magnetism course includes the analysis of different field source distributions through Gauss’s and Ampere’s laws to compute and interpret different physical quantities, such as electric flux, electric and magnetic field, or magnetic circulation. Still, some students have difficulties with these calculations or, in some cases, identifying the differences between those quantities. We present this article to explore and compare the challenges that students experience when asked to compute the electric flux (surface integral of the electric field) or the magnetic circulation (line integral of the magnetic field) in a nonsymmetric field-source distribution with two opposite field sources inside a Gaussian spherical surface or Amperian circular trajectory. The sample consisted of 322 engineering students finishing an electricity and magnetism course. They were presented with two parallel problems. Half answered one in the electricity context and the other in the magnetism context. After a phenomenographic analysis, our results showed that the students’ conceptual difficulties in both contexts can be grouped into the same categories but are not contextually parallel, as has happened when analyzing other electricity and magnetism concepts. Our results also suggest that the magnetic circulation concept is far more unfamiliar to students than the electric flux. We propose several factors that could explain this finding and suggest teaching to address the conceptual difficulties identified in our analysis.

This study investigates the evolution and associations between exam grades and social comparison concern (SCC) among students in an introductory calculus-based physics course. We begin with a descriptive characterization of midterm and final exam scores as well as pre-post SCC scores, including the concurrent evolution of these scores during the course. We hypothesize a feedback loop in which changes in SCC scores are mediated by exam grades, and changes in exam scores are mediated by SCC scores. We employ a structural equation model to determine whether the data are consistent with these hypotheses. Results indicate that there were significant within-student changes in the relative grade standing from exam to exam and that changes in SCC scores depended on both the pre-SCC scores and scores on the first midterm exam. Further, we find evidence that exam scores partially mediate the association between pre- and post-SCC scores, and in turn, post-SCC scores partially mediate associations between midterm and final exam scores, though the mediation effects are somewhat small, comprising 5%–10% of the total effects between exam scores and SCC. We also find that while SCC scores are somewhat correlated with exam scores, they are only very weakly correlated with nonexam grade components, consistent with the idea that exam scores (rather than nonexam scores) are driving changes in SCC and vice versa. Overall, the results provide empirical, correlational evidence to motivate further experimental investigation into a hypothesized dynamic and iterative feedback loop in which student concern about ability or performance compared to others (SCC) can either negatively or positively interfere with student performance on exams.

In this paper, we seek to evaluate the extent to which students can follow a deductive reasoning chain when it is presented to them. A great deal of instruction in introductory physics courses is centered on presenting students with a logical argument that starts from first principles and systematically leads to a particular conclusion. This approach to instruction may conflict with current models of how students reason, including dual-process theories of reasoning and decision making. We investigated student ability to follow reasoning chains at several different institutions, across multiple topics that span introductory mechanics and electricity and magnetism. (For the purposes of this study, we operationally define “following” a reasoning chain—either a correct chain or an incorrect chain—as selecting the appropriate conclusion to a given chain.) To accomplish this, we asked students to answer a physics question and provide an explanation for their answer. We then presented them with a reasoning chain generated by a fictitious student and asked them to select the appropriate concluding statement for that chain. Some of these fictitious chains were fully correct, while others contained a conceptual or logical error. We intentionally used tasks for which students would be unlikely to generate correct reasoning chains on their own. We found that for most tasks, students were generally successful at following chains based on common incorrect reasoning. Students who themselves generated the correct reasoning, however, were much more successful at following correct reasoning chains. Connections between this work and dual-process theories, as well as implications for instruction, are discussed.

Work and mechanical energy is a fundamental topic in introductory physics. Studies in existing literature have shown that students have difficulties in understanding work and mechanical energy, particularly the topic of work-energy theorem. To study students’ knowledge integration in learning work and mechanical energy, a conceptual framework model of work and mechanical energy was developed and applied to guide the design of an assessment for measuring students’ level of knowledge integration. Using the assessment, qualitative and quantitative data were collected in two high schools in an eastern Chinese city. The results reveal that the conceptual framework model can effectively represent the students’ knowledge structures at different levels of knowledge integration. In addition, the assessment is shown effective in identifying unique features of knowledge integration, including context dependence and fragmentation of knowledge components, memorization-based problem-solving strategies, and lack of meaningful connections between work and change in kinetic energy. The conceptual framework of work and mechanical energy and assessment results can provide useful information to facilitate instructional designs to promote knowledge integration.

Women and many people of color continue to be minoritized in STEM and notably in physics. We conducted two studies demonstrating that exposure to counternarratives about who does physics and why one does physics significantly increases high school students—especially women’s—physics-related career intentions. These counternarratives facilitate making connections with students’ career plans and help in sensemaking causes for the continued minoritization of women in physics. Two separate studies measured the impacts of these interventions on students’ physics-related career intentions: first, with an intentionally selected group of teachers (10 teachers, 823 students) across regions and contexts in the U.S.; second, with a randomly sampled group of teachers (13 teachers, 1509 students) from three regions that also included a comparable control group. The results clearly show the importance of exposure to counternarratives in the development of high school students’ career interests, particularly for women and minoritized racial or ethnic groups, and that such counternarratives may help to address systemic issues of underrepresentation in STEM.