The application of factor analysis to the Force Concept Inventory (FCI) has proven to be problematic. Some studies have suggested that factor analysis of test results serves as a helpful tool in assessing the recognition of Newtonian concepts by students. Other work has produced at best ambiguous results. For the FCI administered as a pre- and post-test, we see factor analysis as a tool by which the changes in conceptual associations made by our students may be gauged given the evolution of their response patterns. This analysis allows us to identify and track conceptual linkages, affording us insight as to how our students have matured due to instruction. We report on our analysis of 427 pre- and post-tests. The factor models for the pre- and post-tests are explored and compared along with the methodology by which these models were fit to the data. The post-test factor pattern is more aligned with an expert’s interpretation of the questions’ content, as it allows for a more readily identifiable relationship between factors and physical concepts. We discuss this evolution in the context of approaching the characteristics of an expert with force concepts. Also, we find that certain test items do not significantly contribute to the pre- or post-test factor models and attempt explanations as to why this is so. This may suggest that such questions may not be effective in probing the conceptual understanding of our students.

In this exploratory qualitative study, we describe instructors’ self-reported practices for teaching and assessing students’ ability to troubleshoot in electronics lab courses. We collected audio data from interviews with 20 electronics instructors from 18 institutions that varied by size, selectivity, and other factors. In addition to describing participants’ instructional practices, we characterize their perceptions about the role of troubleshooting in electronics, the importance of the ability to troubleshoot more generally, and what it means for students to be competent troubleshooters. One major finding of this work is that, while almost all instructors in our study said that troubleshooting is an important learning outcome for students in electronics lab courses, only half of instructors said they directly assessed students’ ability to troubleshoot. Based on our findings, we argue that there is a need for research-based instructional materials that attend to both cognitive and noncognitive aspects of troubleshooting proficiency. We also identify several areas for future investigation related to troubleshooting instruction in electronics lab courses.

In this paper we present the results of a research-based teaching-learning sequence on introductory quantum physics based on Feynman’s sum over paths approach in the Italian high school. Our study focuses on students’ understanding of two founding ideas of quantum physics, wave particle duality and the uncertainty principle. In view of recent research reporting the fragmentation of students’ mental models of quantum concepts after initial instruction, we collected and analyzed data using the assessment tools provided by knowledge integration theory. Our results on the group of $n=14$ students who performed the final test indicate that the functional explanation of wave particle duality provided by the sum over paths approach may be effective in leading students to build consistent mental models of quantum objects, and in providing them with a unified perspective on both the photon and the electron. Results on the uncertainty principle are less clear cut, as the improvements over traditional instruction appear less significant. Given the low number of students in the sample, this work should be interpreted as a case study, and we do not attempt to draw definitive conclusions. However, our study suggests that (i) the sum over paths approach may deserve more attention from researchers and educators as a possible route to introduce basic concepts of quantum physics in high school, and (ii) more research should be focused not only on the correctness of students’ mental models on individual concepts, but also on the ability of students to connect different ideas and experiments related to quantum theory in an organized whole.

Constructing and evaluating explanations are important science practices, but in large classes it can be difficult to effectively engage students in these practices and provide feedback. Peer review and grading are scalable instructional approaches that address these concerns, but which raise questions about the validity of the peer grading process. Calibrated Peer Review (CPR) is a web-based system that scaffolds peer evaluation through a “calibration” process where students evaluate sample responses and receive feedback on their evaluations before evaluating their peers. Guided by an activity theory framework, we developed, implemented, and evaluated CPR-based tasks in guided-inquiry, conceptual physics courses for future teachers and general education students. The tasks were developed through iterative testing and revision. Effective tasks had specific and directed prompts and evaluation instructions. Using these tasks, over 350 students at three universities constructed explanations or analyzed physical phenomena, and evaluated their peers’ work. By independently assessing students’ responses, we evaluated the CPR calibration process and compared students’ peer reviews with expert evaluations. On the tasks analyzed, peer scores were equivalent to our independent evaluations. On a written explanation item included on the final exam, students in the courses using CPR outperformed students in similar courses using traditional writing assignments without a peer evaluation element. Our research demonstrates that CPR can be an effective way to explicitly include the science practices of constructing and evaluating explanations into large classes without placing a significant burden on the instructor.

Problem solving is a preferred activity teachers choose to help students learn concepts. At the same time, successful problem solving is widely regarded as a very good indicator of conceptual learning. Many studies have provided evidence that problem solving often improves students’ chances of learning concepts. Still, the question remains relatively unexplored as to how this activity is useful to promote concept learning. In this study we explore this question in the setting of three university students solving a problem on hydrostatics, in which the concept of buoyancy is involved. We use coordination class theory to study how these students progress on their conceptual understanding. We were able to describe how this progress is related to contextual traits, as well as to students’ particular epistemic stances. Finally, we discuss some implications for research and for teaching.

We discuss student participation in an online social annotation forum over two semesters of a flipped, introductory physics course at Harvard University. We find that students who engage in high-level discussion online, especially by providing answers to their peers’ questions, make more gains in conceptual understanding than students who do not. This is true regardless of students’ physics background. We find that we can steer online interaction towards more productive and engaging discussion by seeding the discussion and managing the size of the sections. Seeded sections produce higher quality annotations and a greater proportion of generative threads than unseeded sections. Larger sections produce longer threads; however, beyond a certain section size, the quality of the discussion decreases.

Large-scale audiovisual data that measure group learning are time consuming to collect and analyze. As an initial step towards scaling qualitative classroom observation, we qualitatively coded classroom video using an established coding scheme with and without its audio cues. We find that interrater reliability is as high when using visual data only—without audio—as when using both visual and audio data to code. Also, interrater reliability is high when comparing use of visual and audio data to visual-only data. We see a small bias to code interactions as group discussion when visual and audio data are used compared with video-only data. This work establishes that meaningful educational observation can be made through visual information alone. Further, it suggests that after initial work to create a coding scheme and validate it in each environment, computer-automated visual coding could drastically increase the breadth of qualitative studies and allow for meaningful educational analysis on a far greater scale.

The existing literature indicates that interactive-engagement (IE) based general physics classes improve conceptual learning relative to more traditional lecture-oriented classrooms. Very little research, however, has examined quantitative problem-solving outcomes from IE based relative to traditional lecture-based physics classes. The present study included both pre- and post-course conceptual-learning assessments and a new quantitative physics problem-solving assessment that included three representative conservation of energy problems from a first-semester calculus-based college physics course. Scores for problem translation, plan coherence, solution execution, and evaluation of solution plausibility were extracted for each problem. Over 450 students in three IE-based sections and two traditional lecture sections taught at the same university during the same semester participated. As expected, the IE-based course produced more robust gains on a Force Concept Inventory than did the lecture course. By contrast, when the full sample was considered, gains in quantitative problem solving were significantly greater for lecture than IE-based physics; when students were matched on pre-test scores, there was still no advantage for IE-based physics on gains in quantitative problem solving. Further, the association between performance on the concept inventory and quantitative problem solving was minimal. These results highlight that improved conceptual understanding does not necessarily support improved quantitative physics problem solving, and that the instructional method appears to have less bearing on gains in quantitative problem solving than does the kinds of problems emphasized in the courses and homework and the overlap of these problems to those on the assessment.

Undergraduate research can support students’ more central participation in physics. We analyze markers of two coupled shifts in participation: changes in students’ views about the nature of science coupled to shifts in self-efficacy toward physics research. Students in the study worked with faculty and graduate student mentors on research projects while also participating in a seminar where they learned about research and reflected on their experiences. In classroom discussions and in clinical interviews, students described gaining more nuanced views about the nature of science, specifically related to who can participate in research and what participation in research looks like. This shift was coupled to gains in self-efficacy toward their ability to contribute to research; they felt like their contributions as novices mattered. We present two case studies of students who experienced coupled shifts in self-efficacy and views about nature-of-science shifts, and a case study of a student for whom we did not see either shift, to illustrate both the existence of the coupling and the different ways it can play out. After making the case that this coupling occurs, we discuss some potential underlying mechanisms. Finally, we use these results to argue for more nuanced interpretations of self-efficacy measurements.

In this study, we investigated students’ understanding of concepts related to the microscopic model of gas. We thoroughly reviewed the relevant literature and conducted think alouds with students by asking them to answer open-ended questions about the kinetic molecular theory of gases. Thereafter, we transformed the open-ended questions into multiple-choice questions, whereby distractors were based on the results of the think alouds. Thus, we obtained a set of 22 questions, which constitutes our current version of the kinetic molecular theory of gases concept inventory. The inventory has been administered to 250 students from different universities in Croatia, and its content validity has been investigated trough physics teacher surveys. The results of our study not only corroborate the existence of some already known student misconceptions, but also reveal new insights about a great spectrum of students’ misconceptions that had not been reported in earlier research (e.g., misconceptions about intermolecular potential energy and molecular velocity distribution). Moreover, we identified similar distribution of students’ responses across the surveyed student groups, despite the fact that they had been enrolled in different curricular environments.

Testwiseness is defined as the set of cognitive strategies used by a student that is intended to improve his or her score on a test regardless of the test’s subject matter. Questions with elements that may be affected by testwiseness are common in physics assessments, even in those which have been extensively validated and widely used as evaluation tools in physics education research. The potential effect of several elements of testwiseness were analyzed for questions in the Force Concept Inventory (FCI) and Conceptual Survey on Electricity and Magnetism that contain distractors that are predicted to be influenced by testwiseness. This analysis was performed using data sets collected between fall 2001 and spring 2014 at one midwestern U.S. university (including over 9500 students) and between Spring 2011 and Spring 2015 at a second eastern U.S. university (including over 2500 students). Student avoidance of “none of the above” or “zero” distractors was statistically significant. The effect of the position of a distractor on its likelihood to be selected was also significant. The effects of several potential positive and negative testwiseness effects on student scores were also examined by developing two modified versions of the FCI designed to include additional elements related to testwiseness; testwiseness produced little effect post-instruction in student performance on the modified instruments.

Previous literature on learners’ epistemological beliefs about physics has almost exclusively focused on analysis of university classroom instruction and its effects on students’ views. However, little is known about other populations or factors other than classroom instruction on learners’ epistemologies. In this study, we used a cross-sequential method, combining both longitudinal and cross-sectional designs, to investigate an epistemological progression trend from preservice to in-service teachers. Six cohorts of participants were studied, who either were then attending or had completed an undergraduate teacher preparation program in physics at a major Chinese university. These cohorts were incoming freshmen, end-of-year freshmen, end-of-year sophomores, end-of-year juniors, end-of-year seniors, and 1st-year high school physics teachers who were about to enter the 2nd year of teaching. We used the Colorado Learning Attitudes about Science Survey (CLASS) as both a pretest and a post-test to gauge the changes in the participants’ epistemological views over an entire academic year. Follow-up interviews were also conducted to explore factors responsible for such changes. Results showed that the epistemological trend as measured by CLASS did not increase monotonically. Instead, there was a decrease in the epistemological trend among the incoming freshmen in their first year undergraduate studies, followed by a long stasis until the end of the senior year. Then, there was a rebound for the end-of-year seniors in their 1st year of teaching, followed by another plateau. Interviews revealed that the competitive learning environment, increased content difficulty, and unfamiliar pedagogies in college were major factors that negatively influenced incoming freshmen’s views about physics. Conversely, a role change from student to teacher and relatively easy content in high school positively impacted end-of-year seniors’ views about physics and learning.

Professional development workshops are one of the primary mechanisms used to help faculty improve their teaching, and draw in many STEM instructors every year. Although workshops serve a critical role in changing instructional practices within our community, we rarely assess workshops through careful consideration of how they engage faculty. Initial evidence suggests that workshop leaders often overlook central tenets of education research that are well established in classroom contexts, such as the role of interactivity in enabling student learning [S. Freeman et al., Proc. Natl. Acad. Sci. U.S.A. 111, 8410 (2014)]. As such, there is a need to develop more robust, evidence-based models of how best to support faculty learning in professional development contexts, and to actively support workshop leaders in relating their design decisions to familiar ideas from other educational contexts. In response to these needs, we have developed an observation tool, the real-time professional development observation tool (R-PDOT), to document the form and focus of faculty engagement during workshops. In this paper, we describe the motivation and methodological considerations behind the development of the R-PDOT, justify our decisions to highlight particular aspects of workshop sessions, and demonstrate how the R-PDOT can be used to analyze three sessions from the Physics and Astronomy New Faculty Workshop. We also justify the accessibility and potential utility of the R-PDOT output as a reflective tool using preliminary data from interviews with workshop leaders, and consider the roles the R-PDOT could play in supporting future research on faculty professional development.

This study investigated the multiple-choice test of understanding of vectors (TUV), by applying item response theory (IRT). The difficulty, discriminatory, and guessing parameters of the TUV items were fit with the three-parameter logistic model of IRT, using the parscale program. The TUV ability is an ability parameter, here estimated assuming unidimensionality and local independence. Moreover, all distractors of the TUV were analyzed from item response curves (IRC) that represent simplified IRT. Data were gathered on 2392 science and engineering freshmen, from three universities in Thailand. The results revealed IRT analysis to be useful in assessing the test since its item parameters are independent of the ability parameters. The IRT framework reveals item-level information, and indicates appropriate ability ranges for the test. Moreover, the IRC analysis can be used to assess the effectiveness of the test’s distractors. Both IRT and IRC approaches reveal test characteristics beyond those revealed by the classical analysis methods of tests. Test developers can apply these methods to diagnose and evaluate the features of items at various ability levels of test takers.

Many students struggle with the use of mathematics in physics courses. Although typically well trained in rote mathematical calculation, they often lack the ability to apply their acquired skills to physical contexts. Such student difficulties are particularly apparent in undergraduate electrodynamics, which relies heavily on the use of vector calculus. To gain insight into student reasoning when solving problems involving divergence and curl, we conducted eight semistructured individual student interviews. During these interviews, students discussed the divergence and curl of electromagnetic fields using graphical representations, mathematical calculations, and the differential form of Maxwell’s equations. We observed that while many students attempt to clarify the problem by making a sketch of the electromagnetic field, they struggle to interpret graphical representations of vector fields in terms of divergence and curl. In addition, some students confuse the characteristics of field line diagrams and field vector plots. By interpreting our results within the conceptual blending framework, we show how a lack of conceptual understanding of the vector operators and difficulties with graphical representations can account for an improper understanding of Maxwell’s equations in differential form. Consequently, specific learning materials based on a multiple representation approach are required to clarify Maxwell’s equations.

Just-in-Time Teaching (JiTT) is an instructional strategy involving feedback from students on prelecture activities in order to design in-class activities to build on the continuing feedback from students. We investigate the effectiveness of a JiTT approach, which included in-class concept tests using clickers in an upper-division quantum mechanics course. We analyze student performance on prelecture reading quizzes, in-class clicker questions answered individually, and clicker questions answered after group discussion, and compare those performances with open-ended retention quizzes administered after all instructional activities on the same concepts. In general, compared to the reading quizzes, student performance improved when individual clicker questions were posed after lectures that focused on student difficulties found via electronic feedback. The performance on the clicker questions after group discussion following individual clicker question responses also showed improvement. We discuss some possible reasons for the improved performance at various stages, e.g., from prelecture reading quizzes to postlecture clicker questions, and from individual to group clicker questions.

Improving students’ understanding of the nature of experimental physics is often an explicit or implicit goal of undergraduate laboratory physics courses. However, lab activities in traditional lab courses are typically characterized by highly structured, guided labs that often do not require or encourage students to engage authentically in the process of experimental physics. Alternatively, open-ended laboratory activities can provide a more authentic learning environment by, for example, allowing students to exercise greater autonomy in what and how physical phenomena are investigated. Engaging in authentic practices may be a critical part of improving students’ beliefs around the nature of experimental physics. Here, we investigate the impact of open-ended activities in undergraduate lab courses on students’ epistemologies and expectations about the nature of experimental physics, as well as their confidence and affect, as measured by the Colorado Learning Attitudes about Science Survey for Experimental Physics (E-CLASS). Using a national data set of student responses to the E-CLASS, we find that the inclusion of some open-ended lab activities in a lab course correlates with more expertlike postinstruction responses relative to courses that include only traditional guided lab activities. This finding holds when examining postinstruction E-CLASS scores while controlling for the variance associated with preinstruction scores, course level, student major, and student gender.