We investigate the multilingual and multimodal performance of a large language model-based artificial intelligence (AI) system, GPT-4o, using a diverse set of physics concept inventories spanning multiple languages and subject categories. The inventories, sourced from the PhysPort website, cover classical physics topics such as mechanics, electromagnetism, optics, and thermodynamics, as well as relativity, quantum mechanics, astronomy, mathematics, and laboratory skills. Unlike previous text-only studies, we uploaded the inventories as images to reflect what a student would see on paper, thereby assessing the system’s multimodal functionality. Our results indicate variation in performance across subjects, with laboratory skills standing out as the weakest. We also observe differences across languages, with English and European languages showing the strongest performance. Notably, the relative difficulty of an inventory item is largely independent of the language of the test. When comparing AI results to existing literature on student performance, we find that the AI system outperforms average postinstruction undergraduate students in all subject categories except laboratory skills. Furthermore, the AI performs worse on items requiring visual interpretation of images than on those that are purely text-based. While our exploratory findings show GPT-4o’s potential usefulness in physics education, they highlight the critical need for instructors to foster students’ ability to critically evaluate AI outputs, adapt curricula thoughtfully in response to AI advancements, and address equity concerns associated with AI integration.

Achieving knowledge integration for deep learning requires students to construct well-connected knowledge structures by understanding and applying the central ideas of a concept. However, research on student learning in work and mechanical energy reveals persistent challenges in grasping fundamental concepts, particularly in connecting the work-energy theorem, potential energy, and the principle of conservation of mechanical energy. This study investigates the effectiveness of two instructional interventions based on the conceptual framework of work and mechanical energy: one employing lecture-based instruction and the other incorporating a cooperative learning approach. The findings provide strong evidence that the conceptual framework is an effective tool for guiding instruction to promote knowledge integration. Additionally, the use of cooperative learning instruction was shown to further enhance students’ ability to connect concepts and develop deep understanding, offering valuable insights for designing instructional strategies aimed at fostering knowledge integration.

Previous research has found gender inequitable equipment usage across various lab course contexts. Few studies, however, have tested possible remediation strategies. In this work, we use hierarchical linear modeling to compare men and women’s lab equipment usage in two group work structures across three course contexts. In one in-person course, students formed their own groups in class and rotated into new groups every unit. In the other two courses, one in person and one remote, students were assigned groups formed by the instructor and worked with the same group all semester. In line with former studies, we found gender inequitable equipment usage in the course with in-class formed, rotated groups. We did not observe gender inequitable equipment usage, however, in the course with instructor-assigned, fixed groups. Analyzing equipment usage across the semester within each course, our results suggest that this improvement comes from a combination of both instructor-assigned groups and keeping groups fixed for the semester. Our findings present many opportunities for subsequent controlled studies to probe these practices.

This paper presents a methodological approach based on the use of complex networks to analyze the structure and content of curricula. We analyze the concept network built from the final year of a particular high school physics curriculum, as well as that of mathematics. We examine the most central nodes in each case, the community structures (coherent units or groupings), and the changes that occurred when the network was considered in isolation or integrated with mathematics. The results show that the integrated physics and mathematics network has a higher average degree compared to the individual networks, driven by numerous interdisciplinary connections. The modularity analysis indicates similarities with the original curriculum layout, as well as notable differences that may suggest alternative ways of organizing the content. The differences between separated and integrated networks also highlight the prominence of certain key concepts.

Inequities in student access to trigonometry and calculus are often associated with racial and socioeconomic privilege, and often influence introductory physics course performance. To mitigate these disparities in student preparedness, we developed a two-pronged intervention consisting of (1) incentivized supplemental math assignments and (2) AI-generated learning support tools in the form of optional hints embedded in the physics homework assignments. Both interventions are grounded in the situated expectancy-value theory of achievement motivation, which posits that students are more likely to complete a task that they expect to do well in and whose outcomes they think are valuable. For the supplemental math assignments, the extra credit was scaled to make it worth more points for students with lower exam scores, thereby creating even greater value for students who might benefit most from the assignments. AI-generated hints were integrated into the homework assignments, thereby reducing or eliminating the cost to the student, in terms of time, energy, and social barriers or fear of judgment. Our findings indicate that both these interventions are associated with increased exam scores; in particular, the scaled extra credit reduced disparities in completion of supplemental math assignments. These interventions, which are relatively simple for any instructor to implement, are therefore very promising for creating more equitable undergraduate quantitative courses.

Science, technology, engineering, and mathematics (STEM) graduate and doctoral programs experience consistently high attrition rates. Moreover, persistent disparities exist in gender and racial representation. Students from historically excluded backgrounds have lower retention rates than white and Asian American students. To date, little work has been done to understand the student experience and how it contributes to attrition or persistence. There is also a lack of information regarding the causes of demographic disparities in graduate student attrition. Past studies in this context primarily focused on student academic preparation and mentoring relationships. Although student self-efficacy is found to be a contributing factor for undergraduate student retention, there is currently minimal research on the role of self-efficacy in STEM graduate student retention. This study uses a sociological approach, based on the notion that the surrounding environment influences student experiences and behavior, to understand student retention in physics graduate education. In particular, we examine the relationship between four departmental support structures (mentoring and research experience, professional development, social and academic integration, and financial support) and students’ intentions to persist. We also examine the mediating role of academic self-efficacy in this relationship. We use the construct of self-efficacy under the assumption that self-efficacy is a dynamic construct influenced by, in this case, the academic environment. We use a mixed-method convergent design approach. We collected 396 student responses from 19 physics graduate programs across the United States, 20 follow-up semistructured student interviews, and 9 department leaders’ interviews. This study presents a retention model for graduate education that shows the critical role of student socioacademic integration and self-efficacy on intention to persist. Students’ in-depth interview data corroborate this finding, indicating that a supportive social and academic environment (e.g., working with peers on coursework, socializing, and communicating with faculty members and mentors) helps them in completing the program degree. Moreover, we found that departmental leaders are unaware of the struggles students experience due to the departmental environment (burnout, lack of socioacademic integration, and discrimination) and attribute students’ decisions to leave the program to factors external to the program, such as desire for career change. Student interview data emphasized the importance of cultural change toward supporting student learning and growth, improving work-life balance, and developing and maintaining healthier student-faculty relationships. The results of our study motivate further research to help us better understand how changes in these areas can impact student retention.

Group configurations shape social skills and learning outcomes by fostering student interaction and collaboration. Yet, the factors influencing group formation, social cohesion, and group stability have remained understudied in physics education. Here, we examine these factors in a cohort of 90 students from ninth and tenth grades at a private school in Chile. For a semester-long physics project, students were assigned to groups either through random allocation (control group) or given the autonomy to form their own groups (experimental group). Our findings indicate that the groups formed by choice developed stronger friendship bonds throughout the semester, while the groups formed randomly demonstrated greater levels of cooperation and more frequent changes in membership, suggesting weaker social cohesion. Additionally, tenth-grade students were more likely to stay in their original groups, possibly indicating increased social maturity. These findings emphasize the ongoing challenge in education of balancing student choice in group formation with the need for diverse group compositions that enhance learning. This study provides practical recommendations for physics teachers and researchers.

This study presents the development and validation of a novel multiple-choice test designed to assess university students’ conceptual understanding of electric field, force, and flux. The test of understanding of electric field, force, and flux was constructed based on the results of previous studies using a phenomenographic approach to classify common errors in these concepts. The final 10-item test, designed with targeted distractors, was administered to 116 students enrolled in an introductory electricity and magnetism course. Statistical analyses, including item difficulty and discrimination indices, confirmed the reliability and discriminatory power of the test. Students’ performance revealed significant misconceptions in applying the superposition principle, Gauss’s law, and electric force interactions. The results of this study provide valuable insights into students’ learning challenges and offer physics educators and researchers a useful tool for assessing conceptual understanding and guiding instructional improvements.

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] This study investigates physics instructors’ acceptance and implementation of generative AI (GenAI) in physics education, guided by Rogers’ diffusion of innovation (DOI) theory, focusing on its five-stage innovation-decision process: knowledge, persuasion, decision, implementation, and confirmation. Two survey versions were developed—one for GenAI users and another for nonusers. Data were collected through an online survey featuring five-point Likert scale items and open-ended questions, yielding 320 responses from high school and university physics instructors. The persuasion stage was explored in depth using the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2). A mixed-method approach combining quantitative structural equation modeling (SEM) and qualitative content analysis was employed. We found instructors distributed across different adoption stages, with key barriers including insufficient technical knowledge (51.75% of nonusers) and language processing limitations (50.5% of users). Hedonic motivation ($\beta =0.498$) was found to have a substantially stronger influence on adoption than performance expectancy ($\beta =0.121$), with an effect size approximately 4 times greater. We identified significant differences between paid and free GenAI users, with paid users more likely to request prompt-writing guidance (30.56% vs 6.47%) despite using more advanced models. Support needs also varied substantially, as half of paid users requested budget support for GenAI services compared to only 17.65% of free users. Most physics instructors used GenAI primarily for assessment tasks, particularly for generatingphysics problems with solutions. Their primary concerns included incorrect physics information in GenAI responses, potential negative impacts on students’ analytical thinking, language barriers, and challenges with prompt writing. These findings suggest that successful GenAI integration in physics education requires physics-specific training and differentiated support based on whether instructors use free or paid versions of GenAI tools.

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] Artificial intelligence-based chatbots are increasingly influencing physics education because of their ability to interpret and respond to textual and visual inputs. This study evaluates the performance of two large multimodal model-based chatbots, ChatGPT-4 and ChatGPT-4o, on the brief electricity and magnetism assessment (BEMA), a conceptual physics inventory rich in visual representations such as vector fields, circuit diagrams, and graphs. Quantitative analysis shows that ChatGPT-4o outperforms both ChatGPT-4 and a large sample of university students, and demonstrates improvements in ChatGPT-4o’s vision interpretation ability over its predecessor ChatGPT-4. However, qualitative analysis of ChatGPT-4o’s responses reveals persistent challenges. We identified three types of difficulties in the chatbot’s responses to tasks on BEMA: (i) difficulties with visual interpretation, (ii) difficulties in providing correct physics laws or rules, and (iii) difficulties with spatial coordination and application of physics representations. Spatial reasoning tasks, particularly those requiring the use of the right-hand rule, proved especially problematic. These findings highlight that the most broadly used large multimodal model-based chatbot, ChatGPT-4o, still exhibits significant difficulties in engaging with physics tasks involving visual representations. While the chatbot shows potential for educational applications, including personalized tutoring and accessibility support for students who are blind or have low vision, its limitations necessitate caution. On the other hand, our findings can also be leveraged to design assessments that are difficult for chatbots to solve.

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] We present a study in which a version of a common conservation of mechanical energy introductory physics problem, an object released on an inclined plane, is given to OpenAI’s GPT-4 large language model (LLM). We investigate how different permutations of object, action verb, and property of the incline impact the responses of the LLM. The problem setup and prompting was left purposefully minimal, requiring the LLM to state multiple assumptions to justify the final answer. We specifically studied how different keywords lead the LLM to analyze the system as rolling versus sliding and how this may be different from physics experts and novice learners. We found that domain-specific terminology may impact the LLM differently from students. Even for correct answers, it generally did not specify the necessary assumptions it needed to state to come to this solution, falling short of what would be expected from an expert instructor. When conflicting information was provided, the LLM generally did not indicate that was the case in its responses. Both issues are weaknesses that could be remedied by additional prompting; however, they remain shortcomings in the context of physics teaching. While specific to introductory physics, this study provides insight into how LLMs respond to variations of a problem within a specific topic area and how their strengths and weaknesses may differ from those of humans. Understanding these differences, and tracking them as LLMs change in their capabilities, is crucial for assessing the impact of artificial intelligence on education.

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] Students struggle to acquire the needed energy understanding to meaningfully participate in the energy discourse about socially relevant topics, such as energy transformation or climate change. Identifying students on differing learning trajectories, as well as differences in knowledge used, is essential to help students achieve the needed energy understanding. Collecting and analyzing the longitudinal and fine-grained data necessary for this represents a substantial challenge. However, the use of a digital workbook, which captures all interaction data, has enabled us to collect such data from $N=548$ students (data from 172 students were analyzed after applying exclusion criteria). Using machine learning and natural language processing, we analyzed the data to identify productive and unproductive learning trajectories and their underlying reasons. The learning trajectories were classified according to the post-test score. To analyze the tasks from the digital workbook, machine learning methods, specifically random forest, and natural language processing, were employed to identify how students on different learning trajectories progress through the unit. The random forest analysis was accurate in distinguishing between productive and unproductive learning trajectories. Furthermore, natural language processing was employed to analyze open-ended responses, which revealed disparities in the knowledge elements that students on productive and unproductive trajectories utilized. The findings of this study indicate that machine learning techniques have the potential to provide valuable insights into student learning trajectories, which can inform the design of instructional units and the feedback provided to teachers and students.

Digital simulations are powerful instructional tools for physics education. They are often designed to visualize canonical physical phenomena, with adjustable parameters for influencing the system. While this is sufficient for developing conceptual and qualitative intuitions, it does little to help physics students build connections between physical systems and the mathematical models and equations that represent them. We present PhysMath, a suite of interactive physics simulations for use in upper-division courses. These simulations allow students to explore connections between mathematical equations and the phenomena they represent by inputting, modifying, and observing changes in system behavior. In this paper, we describe our first simulation—the Bead-On-Hoop for Classical Mechanics—and report findings from pilot interviews with intermediate physics students interacting with the simulation. Our findings validate the simulations’ design and highlight its potential for scaffolding students’ mathematical sensemaking.

In 2017, the University of Utah experienced a tragedy that accelerated already proposed changes to the physics and astronomy graduate program. This article outlines the changes made to the graduate program and discusses students’ experiences with the early implementation of the new policies. In-depth, open-ended interviews were conducted with students over 2 years to understand their experiences under or with the new graduate program. Findings suggest that student interviews and direct recruitment aided in students’ decisions to attend a graduate program, and a weeklong orientation helped build community among the cohorts. A centralized Advising Day, dictated by the department, facilitated enrolled students meeting with department members to ensure they were on track in their program and taking courses geared toward research interests favored by students. Findings suggest that qualifying or comprehensive exams need more instruction and study materials, and that students still face issues of financial insecurities, healthcare concerns, and sexism that need addressing. Further research should examine how these reforms have impacted students at the end of their time in the program. Content Warning: This paper discusses the death by suicide of a graduate student.

[This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] Recent research in science education has largely focused on using ChatGPT to solve problems and evaluating its accuracy and problem-solving features. However, as artificial intelligence (AI) is becoming an important tool for human development, effective strategies for learning and education using human-AI collaboration (HAI) learning remain underexplored. HAI is emerging as a powerful tool for enhancing educational outcomes, especially with the integration of advanced natural language AI systems such as ChatGPT-4o. This study explores the role of HAI in solving scientific problems, comparing its efficacy with human-human collaboration (HHC) among high school students. We employed two different groups: one using HAI with ChatGPT-4o, and the other using HHC, focusing on their performance in scientific knowledge and scientific thinking tasks. Results showed that while both collaboration methods significantly improved students’ performance in solving scientific problems, HHC exhibited a greater effect size compared to HAI. The use of ChatGPT-4o was particularly noted for its ability to provide interactive learning experiences and guide students through problem solving; however, challenges such as inconsistencies in responses and image recognition hindered its effectiveness. Additionally, students tended to use ChatGPT-4o as a tool for obtaining answers rather than engaging in deeper collaborative explorations. This study underscores the importance of developing targeted training for students in effectively collaborating with AI tools. The results can also provide useful information for future exploration on methods to strengthen HHC by integrating AI tools, leveraging the strengths of both approaches to enhance education outcomes.

There is currently little physics education literature examining thinking and learning in graduate education and even less literature characterizing problem solving among physics graduate students despite this being an essential professional skill for physicists. Given reports of discrepancies between physics problem solving in the undergraduate classroom and “real-world” problem solving, we sought to investigate whether this discrepancy exists at the graduate level. We first investigate the problem-solving skills present in first-year graduate physics assignments. A recent framework that characterizes problem solving as decisions-to-be-made was used. Assignments were taken from the four core courses of one academic year at one research-intensive university and coded by two researchers. We found that only 4 of the 29 decisions in the framework were present in most of the assignments. We then interviewed 11 instructors from 3 universities and asked which decisions they expected of first-year graduate students. Eleven decisions were expected by eight or more of the participants, but only four of these decisions were commonly practiced on assignments. Therefore, there seems to be a mismatch between instructor expectations and practice of problem solving on assignments. This suggests that graduate physics courses may not be aligned with the problem-solving skills that physics graduate students will need in their research or future careers.

This work explores the integration of artificial intelligence (AI) custom chatbots in educational settings, with a particular focus on their applicability in the context of mathematics and physics. In view of the increasing deployment of AI tools such as ChatGPT in educational contexts, the present study explores their potential in generating topic-related learning material. The study assesses the impact of learning with AI-generated explanations as Supplemental Material on the learning experiences and performance of sixth-grade students, with a particular focus on proportional relationships in mathematical and physical contexts. The randomized controlled study with $N=214$ students compared supplementary learning material in the form of traditional textbook material with explanations previously generated by an AI custom chatbot. The results demonstrated that while the AI-generated materials had an indefinite impact on learning outcomes, they significantly enhanced positive-activating emotions, situational interest, and self-efficacy while reducing intrinsic and extrinsic cognitive load. These findings underscore the potential of AI to transform educational practices by fostering a superior learning experience. However, further research is required to clarify its impact on learning performance and long-term learning outcomes. The study highlights the importance of careful integration and customization of AI tools to maximize their benefits in physics education.

In this quantitative study, we explored learners’ intrinsic motivation towards three different experimental physics educational activities offered by our department to upper-grade high school students. The inspected activities were lecture demonstrations, a science show, and hands-on practical work. The respondents were Czech high school students ($N=9647$) who participated in these activities. From each of these activities, a highly comparable subset of 1128 respondents was selected for the comparative analysis. To assess their intrinsic motivation and its covariates, we used three dimensions of the Intrinsic Motivation Inventory: interest/enjoyment, effort/importance, and value/usefulness. The data were evaluated using a unidimensional IRT Graded Response model. Our study revealed that the science show was more appealing to high school students than traditional lecture demonstrations and, surprisingly, even more appealing than practical lab work. This increased interest was more pronounced among students who identified themselves as less diligent in physics. Paradoxically, the respondents perceived the science show as less useful than the other two activities. Delving deeper, gender differences in perceived intrinsic motivation were generally minor, with one exception: in a comparison of the interest generated by the science show and practical work, girls favored the science show more than boys did. These findings underscore the need for a nuanced understanding of pedagogical strategies and offer insights into different approaches to promoting student engagement and interest in physics. Our research could assist educators in distinguishing the activities that might be more beneficial in terms of generating interest among particular groups of students.

Teachers’ goals and orientations underlie their instructional decision making and can make the difference between “talking the talk” and “walking the walk” when trying to change lab instruction. Schoenfeld’s Resources, Orientations, and Goals (ROG) framework was employed to shed light on the challenges involved in a reform in lab instruction that aims to grant students more agency in experimental research practices. We focus on high school physics teachers engaged in a professional development process as members of a national network of Professional Learning Communities (PLCs, $N\approx 275$ participants). One of the annual themes in the PLCs addressed lab instruction. The teachers were involved in: (a) experiencing, as learners, Deliberation Labs designed to reform lab instruction while taking into consideration the widespread constraints faced by high school teachers; and (b) engaging in collaborative reflection on their classroom experiences to encourage them to rethink their ROGs. The data were gathered from two professional development sequences held during the 2020 and 2022 school years that differed in the structuring of collaborative reflection. To achieve a large-scale portrayal of teachers’ goals and orientations toward integrating experimental research practices in the instructional lab, we analyzed surveys administered in the PLCs. The survey results highlighted two main orientations that interfered when implementing the Deliberation Labs: the “lab serving theory” orientation and the “restricting agency” orientation. To better understand the processes of change in teachers’ ROGs, we present two case studies that were involved in the structured collaborative reflection on practice. They deal with experienced PLC teacher-leaders who were motivated to try out the Deliberation Labs in their classrooms. The analysis shows how they interpreted their experiences in ways that align with the lab serves theory orientation, thus undermining the goals that the Deliberation Labs were designed to achieve. These results highlight the winding road to change in teachers’ ROGs and emphasize the need to support them in resolving conflicting goals and orientations. They also raise dilemmas regarding the nature of feasible yet effective professional development programs that can lead to large-scale change.