PeerWise is an online application where students are encouraged to generate a bank of multiple choice questions for their classmates to answer. After answering a question, students can provide feedback to the question author about the quality of the question and the question author can respond to this. Student use of, and attitudes to, this online community within PeerWise was investigated in two large first year undergraduate physics courses, across three academic years, to explore how students interact with the system and the extent to which they believe PeerWise to be useful to their learning. Most students recognized that there is value in engaging with PeerWise, and many students engaged deeply with the system, thinking critically about the quality of their submissions and reflecting on feedback provided to them. Students also valued the breadth of topics and level of difficulty offered by the questions, recognized the revision benefits afforded by the resource, and were often willing to contribute to the community by providing additional explanations and engaging in discussion.

We explore the sources of student engagement with curricular content in an Introductory Physics for Life Science (IPLS) course at Swarthmore College. Do IPLS students find some life-science contexts more interesting than others, and, if so, what are the sources of these differences? We draw on three sources of student data to answer this question: (1) quantitative survey data illustrating how interested students were in particular contexts from the curriculum, (2) qualitative survey data in which students describe the source of their interest in these particular contexts, and (3) interview data in which students reflect on the contexts that were and were not of interest to them. We find that examples that make interdisciplinary connections with students’ other coursework in biology and chemistry, and examples that make connections to what students perceive to be the “real world,” are particularly effective at fostering interest. More generally, students describe being deeply engaged with contexts that foster a sense of coherence or have personal meaning to them. We identify various “engagement pathways” by which different life-science students engage with IPLS content, and suggest that a curriculum needs to be flexible enough to facilitate these different pathways.

The Conceptual Survey of Electricity and Magnetism (CSEM) has been used to assess student understanding of introductory concepts of electricity and magnetism because many of the items on the CSEM have strong distractor choices which correspond to students’ alternate conceptions. Instruction is unlikely to be effective if instructors do not know the common alternate conceptions of introductory physics students and explicitly take into account common student difficulties in their instructional design. Here, we discuss research involving the CSEM to evaluate one aspect of the pedagogical content knowledge of teaching assistants (TAs): knowledge of introductory students’ alternate conceptions in electricity and magnetism as revealed by the CSEM. For each item on the CSEM, the TAs were asked to identify the most common incorrect answer choice selected by introductory physics students if they did not know the correct answer after traditional instruction. Then, we used introductory student CSEM post-test data to assess the extent to which TAs were able to identify the most common alternate conception of introductory students in each question on the CSEM. We find that the TAs were thoughtful when attempting to identify common student difficulties and they enjoyed learning about student difficulties this way. However, they struggled to identify many common difficulties of introductory students that persist after traditional instruction. We discuss specific alternate conceptions that persist after traditional instruction, the extent to which TAs were able to identify them, and results from think-aloud interviews with TAs which provided valuable information regarding why the TAs sometimes selected certain alternate conceptions as the most common but were instead very rare among introductory students. We also discuss how tasks such as the one used in this study can be used in professional development programs to engender productive discussions about the importance of being knowledgeable about student alternate conceptions in order to help students learn. Interviews with TAs engaged in this task as well as our experience with such tasks in our professional development programs suggest that they are beneficial.

Relating mathematical concepts to graphical representations is a challenging task for students. In this paper, we introduce two visual strategies to qualitatively interpret the divergence of graphical vector field representations. One strategy is based on the graphical interpretation of partial derivatives, while the other is based on the flux concept. We test the effectiveness of both strategies in an instruction-based eye-tracking study with $N=41$ physics majors. We found that students’ performance improved when both strategies were introduced (74% correct) instead of only one strategy (64% correct), and students performed best when they were free to choose between the two strategies (88% correct). This finding supports the idea of introducing multiple representations of a physical concept to foster student understanding. Relevant eye-tracking measures demonstrate that both strategies imply different visual processing of the vector field plots, therefore reflecting conceptual differences between the strategies. Advanced analysis methods further reveal significant differences in eye movements between the best and worst performing students. For instance, the best students performed predominantly horizontal and vertical saccades, indicating correct interpretation of partial derivatives. They also focused on smaller regions when they balanced positive and negative flux. This mixed-method research leads to new insights into student visual processing of vector field representations, highlights the advantages and limitations of eye-tracking methodologies in this context, and discusses implications for teaching and for future research. The introduction of saccadic direction analysis expands traditional methods, and shows the potential to discover new insights into student understanding and learning difficulties.

Measuring student learning is a complicated but necessary task for understanding the effectiveness of instruction and issues of equity in college science, technology, engineering, and mathematics (STEM) courses. Our investigation focused on the implications on claims about student learning that result from choosing between one of two commonly used metrics for analyzing shifts in concept inventories. The metrics are normalized gain ($\mathbf{g}$), which is the most common method used in physics education research and other discipline based education research fields, and Cohen’s $d$, which is broadly used in education research and many other fields. Data for the analyses came from the Learning About STEM Student Outcomes (LASSO) database and included test scores from 4551 students on physics, chemistry, biology, and math concept inventories from 89 courses at 17 institutions from across the United States. We compared the two metrics across all the concept inventories. The results showed that the two metrics lead to different inferences about student learning and equity due to the finding that $\mathbf{g}$ is biased in favor of high pretest populations. We discuss recommendations for the analysis and reporting of findings on student learning data.

Drawing appropriate diagrams is a useful problem solving heuristic that can transform a problem into a representation that is easier to exploit for solving it. One major focus while helping introductory physics students learn effective problem solving is to help them understand that drawing diagrams can facilitate problem solution. We conducted an investigation in which two different interventions were implemented during recitation quizzes in a large enrollment algebra-based introductory physics course. Students were either (i) asked to solve problems in which the diagrams were drawn for them or (ii) explicitly told to draw a diagram. A comparison group was not given any instruction regarding diagrams. We developed rubrics to score the problem solving performance of students in different intervention groups and investigated ten problems. We found that students who were provided diagrams never performed better and actually performed worse than the other students on three problems, one involving standing sound waves in a tube (discussed elsewhere) and two problems in electricity which we focus on here. These two problems were the only problems in electricity that involved considerations of initial and final conditions, which may partly account for why students provided with diagrams performed significantly worse than students who were not provided with diagrams. In order to explore potential reasons for this finding, we conducted interviews with students and found that some students provided with diagrams may have spent less time on the conceptual analysis and planning stage of the problem solving process. In particular, those provided with the diagram were more likely to jump into the implementation stage of problem solving early without fully analyzing and understanding the problem, which can increase the likelihood of mistakes in solutions.

A growing body of scholarly work indicates that student performance on physics problems stems from many factors, including relevant conceptual understanding. However, in contexts in which significant conceptual difficulties have been documented via research, it can be difficult to pinpoint and isolate such factors because students’ written and interview responses rarely reveal the full richness of their conscious and, perhaps more importantly, subconscious reasoning paths. In this investigation, informed by dual-process theories of reasoning and decision making as well as the theoretical construct of accessibility, we conducted a series of experiments in order to gain greater insight into the factors impacting student performance on the “five-block problem,” which has been used in the literature to probe student thinking about buoyancy. In particular, we examined both the impact of problem design (including salient features and cueing) and the impact of targeted instruction focused on density-based arguments for sinking and floating and on neutral buoyancy. The investigation found that instructional modifications designed to remove the strong intuitive appeal of the first-available response led to significantly improved performance, without improving student conceptual understanding of the requisite buoyancy concepts. As such, our findings represent an important first step in identifying systematic strategies for using theories from cognitive science to guide the development and refinement of research-based instructional materials.

In this study, we analyze the systematic error from false positives of the Force Concept Inventory (FCI). We compare the systematic errors of question 6 (Q.6), Q.7, and Q.16, for which clearly erroneous reasoning has been found, with Q.5, for which clearly erroneous reasoning has not been found. We determine whether or not a correct response to a given FCI question is a false positive using subquestions. In addition to the 30 original questions, subquestions were introduced for Q.5, Q.6, Q.7, and Q.16. This modified version of the FCI was administered to 1145 university students in Japan from 2015 to 2017. In this paper, we discuss our finding that the systematic errors of Q.6, Q.7, and Q.16 are much larger than that of Q.5 for students with mid-level FCI scores. Furthermore, we find that, averaged over the data sample, the sum of the false positives from Q.5, Q.6, Q.7, and Q.16 is about 10% of the FCI score of a midlevel student.

In response to national calls for improved physical sciences education for students pursuing careers in the life sciences and medicine, reformed introductory physics for life sciences (IPLS) courses are being developed. This exploratory study is among the first to assess the effect of an IPLS course on students’ attitudes, interest, and performance. The IPLS course studied was the second semester of introductory physics, following a standard first semester course, allowing the outcomes of the same students in a standard course and in an IPLS course to be compared. In the IPLS course, each physics topic was introduced and elaborated in the context of a life science example, and developing students’ skills in applying physics to life science situations was an explicitly stated course goal. Items from the Colorado Learning about Science Survey were used to assess change in students’ attitudes toward and their interest in physics. Whereas the same students’ attitudes declined during the standard first semester course, we found that students’ attitudes toward physics hold steady or improve in the IPLS course. In particular, students with low initial interest in physics displayed greater increases in both attitudes and interest during the IPLS course than in the preceding standard course. We also find that in the IPLS course, students’ interest in the life science examples is a better predictor of their performance than their pre-IPLS interest in physics. Our work suggests that the life science examples in the IPLS course can support the development of student interest in physics and positively influence their performance.

This article discusses our investigation regarding nature of science (NOS) implementations and epistemological beliefs within an undergraduate introductory astronomy course. The five year study consists of two years of baseline data in which no explicit use of NOS material was implemented, then three years of subsequent data in which specific NOS material was integrated into the classroom. Our original study covered two years of baseline data and one year of treatment data. Two additional years of treatment course data have revealed intriguing new insights into our students’ epistemic belief structure. To monitor the evolution of belief structures across each semester we used student pre-post data on the Epistemological Beliefs About the Physical Sciences (EBAPS) assessment. The collected data were also partitioned and analyzed according to the following variables: college (Letters of Science, Business, Education, etc.), degree (BA or BS), status (freshman, sophomore, etc.), and gender (male or female). We find that the treatment course no longer undergoes significant overall epistemic deterioration after a semester of instruction. We also acquire a more detailed analysis of these findings utilizing the aforementioned variables. Most notably, we see that this intervention had a pronounced positive impact on males and on students within the college of Education, Arts & Architecture, and those with no concentration. Lastly, whether or not students believe their ability to learn science is innate or malleable did not seem to change, remaining a rigid construct with student epistemologies.

Although active learning is supported by strong evidence of efficacy in undergraduate science instruction, institutions of higher education have yet to embrace comprehensive change. Costs of transforming instruction are regularly cited as a key factor in not adopting active-learning instructional practices. Some cite that alternative methods to stadium-style, lecture-based education are not financially viable to an academic department. This paper examines that argument by presenting an ingredients approach to estimating costs of two instructional methods used in introductory university physics courses at a large public U.S. university. We use a metric common in educational economics, cost effectiveness (CE), which is the total cost per student passing the class. We then compare the CE of traditional, passive-learning lecture courses to those of a well-studied, active-learning curriculum (Modeling Instruction) as a way of evaluating the claim that active learning is cost prohibitive. Our findings are that the Modeling Instruction approach has a higher cost per passing student ($\mathrm{MI}=\$1,030/\mathrm{passing}$ student vs $\mathrm{Trad}=\$790/\mathrm{passing}$ student). These results are discussed from perspectives of university administrators, students, and taxpayers. We consider how MI would need to adapt in order to make the benefits of active learning (particularly higher pass rates and gains on multiple measured student outcomes) available in a cost-neutral setting. This approach aims to provide a methodology to better inform decision makers balancing financial, personnel, and curricular considerations.

This paper investigates the efficacy of an open-inquiry approach to achieve a long term stability of physics instruction. This study represents the natural continuation of a research project started four years ago when a sample of thirty engineering undergraduates, having already attended traditional university physics instruction, were involved in a six-week long learning experience of open-inquiry research activities within the highly motivating context of developing a thermodynamically efficient space base on Mars. Four years later, we explore the effectiveness of that learning experience by analyzing the outcomes that the students achieved by answering again the same questionnaire that was administered them both prior to and immediately after those activities. As we did in the first work, students’ answers were classified within three epistemological profiles. Now, a comparison among students’ outcomes during the three phases, namely, preinstruction, postinstruction, and after four years has been carried out. Immediately after the open-inquiry experience, the students obtained significant benefits in terms of the strengthening of their practical and reasoning abilities, by proficiently applying the learned concepts to face and solve real-world problem situations. In this study, the students’ answers do not highlight any significant regress towards their preinstruction profiles. The global robustness of the teaching strategy adopted four years ago is confirmed by a statistically significant comparison with a control group of students who experienced the same curricular instruction except for the open inquiry-based workshop. Nevertheless, some changes have been observed and discussed in the light of the answers the students provided to a short interview regarding their studying or working experiences across the four-year temporal window.

Student attitudes, defined as the extent to which one holds expertlike beliefs about and approaches to physics, are a major research topic in physics education research. An implicit but rarely tested assumption underlying much of this research is that student attitudes play a significant part in student learning and performance. The current study directly tested this attitude-learning link by measuring the association between incoming attitudes (Colorado Learning Attitudes about Science Survey) and student learning during the semester after statistically controlling for the effects of prior knowledge [early-semester Force Concept Inventory (FCI) or Brief Electricity and Magnetism Assessment (BEMA)]. This study spanned four different courses and included two complementary measures of student knowledge: late-semester concept inventory scores (FCI or BEMA) and exam averages. In three of the four courses, after controlling for prior knowledge, attitudes significantly predicted both late-semester concept inventory scores and exam averages, but in all cases these attitudes explained only a small amount of variance in concept-inventory and exam scores. Results indicate that after accounting for students’ incoming knowledge, attitudes may uniquely but modestly relate to how much students learn and how well they perform in the course.

The Force Concept Inventory was designed to poll the Newtonian conception of force. While there are many in-depth studies analyzing response data that look at the structure of the correct answers, we believe that the incorrect answers also carry revealing information about the students’ worldview. The inventory was originally designed so that the “distractors” in each question reflected commonly held misconceptions, and thus the rate at which students guess the correct answer is very low. Students select incorrect answers that correspond to the misconception that they hold and there are very few responses which appear obviously wrong to students. A side effect of this approach is that the incorrect responses reflect the non-Newtonian worldviews held by students. These non-Newtonian worldviews are coherent and robust, and this, at least in part, helps to explain why these misconceptions are so resistant to instruction. In this study we focus once more on the misconception data in Force Concept Inventory responses, particularly on the linkages between these misconceptions. We hypothesize that there are distinct groupings of distractor items formed by the strength of the association between these items. The two largest groupings are associated with the “impetus” world view in which the motion of an object is determined by the quantity of impetus which that object contains. We find that certain central items hold particularly important places in these groupings and also that individual groupings may be connected to each other by “connector” items. We finally suggest that, on the basis of this study, that these non-Newtonian worldviews might best be dismantled by addressing these key central and connector items.

In the context of introductory physics, we study student conceptual understanding of differentials, differential products, and integrals and possible pathways to understanding these quantities. We developed a multiple choice conceptual assessment employing a variety of physical contexts probing physical understanding of these three quantities and administered the instrument to over 1000 students in first and second semester introductory physics courses. Using a regression-based mediation analysis with conceptual understanding of integration as the dependent variable, we found evidence consistent with a simple mediation model: the relationship between differentials scores and integral scores may be mediated by the understanding of differential products. The indirect effect (a quantifiable metric of mediation) was estimated as $ab=0.29$, 95% CI [0.25, 0.33] for $N=1102$ Physics 1 students, and $ab=0.27$, 95% CI [0.14, 0.48] for $N=65$ Physics 2 students. We also find evidence that the physical context of the questions can be an important factor. These results imply that for introductory physics courses, instructional emphasis first on differentials then on differential products in a variety of contexts may in turn promote better integral understanding.

In the context of a generic harmonic oscillator, we investigated students’ accuracy in determining the period, frequency, and angular frequency from mathematical and graphical representations. In a series of studies including interviews, free response tests, and multiple choice tests developed in an iterative process, we assessed students in both algebra-based and calculus-based, traditionally instructed university-level introductory physics courses. Using the results, we categorized nine skills necessary for proficiency in determining period, frequency, and angular frequency. Overall results reveal that, postinstruction, proficiency is quite low: only about 20%–40% of students mastered most of the nine skills. Next, we used a semiquantitative, intuitive method to investigate the hierarchical structure of the nine skills. We also employed the more formal item tree analysis method to verify this structure and found that the skills form a multilevel, nonlinear hierarchy, with mastery of some skills being prerequisite for mastery in other skills. Finally, we implemented a targeted, 30-min group-work activity to improve proficiency in these skills and found a 1 standard deviation gain in accuracy. Overall, the results suggest that many students currently lack these essential skills, targeted practice may lead to required mastery, and that the observed hierarchical structure in the skills suggests that instruction should especially attend to the skills lower in the hierarchy.

Research on the test structure of the Force Concept Inventory (FCI) has largely ignored gender, and research on FCI gender effects (often reported as “gender gaps”) has seldom interrogated the structure of the test. These rarely crossed streams of research leave open the possibility that the FCI may not be structurally valid across genders, particularly since many reported results come from calculus-based courses where 75% or more of the students are men. We examine the FCI considering both psychometrics and gender disaggregation (while acknowledging this as a binary simplification), and find several problematic questions whose removal decreases the apparent gender gap. We analyze three samples (total $N_{\mathrm{pre}}=5391$, $N_{\mathrm{post}}=5769$) looking for gender asymmetries using classical test theory, item response theory, and differential item functioning. The combination of these methods highlights six items that appear substantially unfair to women and two items biased in favor of women. No single physical concept or prior experience unifies these questions, but they are broadly consistent with problematic items identified in previous research. Removing all significantly gender-unfair items halves the gender gap in the main sample in this study. We recommend that instructors using the FCI report the reduced-instrument score as well as the 30-item score, and that credit or other benefits to students not be assigned using the biased items.

We describe the construction, validation, and testing of a concept inventory for an Introduction to Physics of Semiconductors course offered by the department of physics to undergraduate engineering students. By design, this inventory addresses both content knowledge and the ability to interpret content via different cognitive processes outlined in Bloom’s revised taxonomy. The primary challenge comes from the low number of test takers. We describe the Rasch modeling analysis for this concept inventory, and the results of the calibration on a small sample size, with the intention of providing a useful blueprint to other instructors. Our study involved 101 students from Oklahoma State University and fourteen faculty teaching or doing research in the field of semiconductors at seven universities. The items were written in four-option multiple-choice format. It was possible to calibrate a 30-item unidimensional scale precisely enough to characterize the student population enrolled each semester and, therefore, to allow the tailoring of the learning activities of each class. We show that this scale can be employed as an item bank from which instructors could extract short testlets and where we can add new items fitting the existing calibration.

Group work is becoming increasingly common in introductory physics classrooms. Understanding how students engage in these group learning environments is important for designing and facilitating productive learning opportunities for students. We conducted a study in which we collected video of groups of students working on conceptual electricity and magnetism problems in an introductory physics course. In this setting, students needed to negotiate a common understanding and coordinate group decisions in order to complete the activity successfully. We observed students interacting in several distinct ways while solving these problems. Analysis of these observations focused on identifying the different ways students interacted and articulating what defines and distinguishes them, resulting in the development of the modes of collaboration framework. The modes of collaboration framework defines student interactions along three dimensions: social, discursive, and disciplinary content. This multidimensional approach offers a unique lens through which to consider group work and provides a flexibility that could allow the framework to be adapted for a variety of contexts. We present the framework and several examples of its application here.