In science, technology, engineering, and mathematics education there has been increased emphasis on teaching goals that include not only the learning of content knowledge but also the development of scientific reasoning skills. The Lawson classroom test of scientific reasoning (LCTSR) is a popular assessment instrument for scientific reasoning. Through large scale applications, however, several issues have been observed regarding the validity of the LCTSR. This paper will review the literature on the assessment of scientific reasoning and provide a detailed analysis of the current version of LCTSR in regards to its validity and measurement features. The results suggest that the LCTSR is a practical tool for assessing a unidimensional scale of scientific reasoning. The instrument has a good overall reliability for the whole test. However, inspections of individual question pairs reveal a variety of validity concerns for five of the total twelve question pairs. These five question pairs have a considerable amount of inconsistent response patterns. Qualitative analysis also indicates wide ranging question design issues. Therefore, assessment of subskills involved with the five question pairs may have less power due to their questionable validity.

This study reports on the development, validation, and administration of a 48-item multiple-choice test to assess students’ representational fluency of linear functions in a physics context (1D kinematics) and a mathematics context. The test includes three external representations: graphs, tables, and formulas, which result in six possible representational transitions between them. Moreover, four linear function types are included: negative $y$ intercept and positive slope, zero $y$ intercept and positive slope, positive $y$ intercept and negative slope, and positive $y$ intercept and positive slope. The test is administered to 385 students aged 14–15 in the 9th grade from 13 schools in Flanders (Belgium) and—after validation—is analyzed by means of generalized estimating equations (GEE). The results show a significant main effect for all design factors and a significant interaction effect between representational transition and function type, as well as additional interaction effects with the gender of the respondents. Furthermore, representational transitions which include a formula prove to be significantly more difficult; in particular for the directly proportional function type, the transition to a formula stands out from the analysis. Mean accuracies in physics are significantly lower compared to mathematics. Function types with negative values for either $y$ intercept or slope also result in significantly lower mean accuracies indicating the difficulty students have with negative numbers in linear functions. A distractor analysis of the incorrect answers chosen by the students reveals three distinct dominant errors: concept switching, sign switching, and switching of the directly proportional function type and the function type with positive $y$ intercept and positive slope.

Assessment instruments composed of two-tier multiple choice (TTMC) items are widely used in science education as an effective method to evaluate students’ sophisticated understanding. In practice, however, there are often concerns regarding the common scoring methods of TTMC items, which include pair scoring and individual scoring schemes. The pair-scoring method is effective in suppressing “false positives” at the cost of missing possible middle states of progression of student understanding. On the other hand, the individual scoring method captures an undistinguished middle level but is prone to rewarding guessing, which leads to “false positives”. In addition, this middle level does not discriminate the progression between knowing the result and explaining the reason, which limits the capacity of drawing meaningful implications from the assessment outcomes. To address the concerns with the current scoring methods, it is valuable to explore new scoring method(s) that can fully utilize the information measured with TTMC items. In this study, a number of scoring models are studied using Rasch analysis on data of a popular TTMC test, the Lawson classroom test of scientific reasoning (LCTSR), collected from four considerably different populations. The results show that the model fit quality of the scoring methods varies with student population and item design. In general, there is no one-fits-all solution; however, given the new information obtained in this study, a three-step process is suggested that can guide the development of new mixed scoring models tailored for a particular population and or test. The evaluation results show that the mixed models produce the most reliable model fitting and better than average goodness of fit. Furthermore, the results in this study also confirm previous studies, which suggest that it is harder to come up with a correct explanation than to just know the answer.

Gender gaps on the most widely used conceptual inventories created by physics education researchers have been extensively studied. Most of the research exploring the consistent gender gaps has been performed at the student level using the total evaluation score; less research has been performed examining these assessments at the item level and this research has been predominately restricted to the Force Concept Inventory (FCI). Many studies have identified subsets of FCI items as unfair to either men or women. An item is fair if men and women of equal ability in conceptual physics score equally on the item. This study explored the item-level gender fairness of the Force and Motion Conceptual Evaluation (FMCE) and the Conceptual Survey of Electricity and Magnetism (CSEM). Classical test theory and differential item functioning (DIF) analysis were employed to examine item fairness. Fairness was investigated with four large post-test samples, two for the FMCE ($n_1=3016$, $n_2=3719$) and two for the CSEM ($n_1=2014$, $n_2=2657$). Men and women performed significantly differently on the majority of FMCE items but with no more than a small effect size. There were fewer items in the CSEM where men and women performed differently. Using DIF analysis, which assumes that overall test score is an accurate measure of ability, only one item in the FMCE demonstrated large DIF in both samples with that item unfair to women. One additional item showed large DIF in a single sample, also unfair to women. Only one item in the CSEM demonstrated large DIF. The item was unfair to men but this result was not consistent across all samples. The number of large DIF items identified in both the FMCE and the CSEM was substantially smaller than the number of large DIF items identified in the FCI by previous studies.

The NASA/IPAC Teacher Archive Research Program (NITARP) provides a year-long authentic astronomy research project by partnering a research astronomer with small groups of educators. NITARP has worked with a total of 103 educators since 2005. In this paper, surveys are explored that were obtained from 74 different educators, at up to four waypoints during the course of 13 months, from the class of 2010 through the class of 2017; those surveys reveal how educator participants describe the major changes and outcomes in themselves fostered by NITARP. Three-quarters of the educators self-report some or major changes in their understanding of the nature of science. The program provides educators with experience collaborating with astronomers and other educators, and forges a strong link to the astronomical research community; the NITARP community of practice encourages and reinforces these linkages. During the experience, educators get comfortable with learning complex new concepts, with $\sim 40\%$ noting in their surveys that their approach to learning has changed. Educators are provided opportunities for professional growth; at least 12% have changed career paths substantially in part due to the program, and 14% report that the experience was “life changing.” At least 60% express a desire to include richer, more authentic science activities in their classrooms. This work illuminates what benefits the program brings to its participants; the NITARP approach could be mirrored in similar professional development programs in other STEM subjects.

Federal legislation specifies equitable access to education for all students at all levels of education, including postsecondary. To explore how well the physics education research (PER) community is currently serving students who inherently vary in needs, abilities, and interests, four research-based curricula (Tutorials in Introductory Physics, Open Source Tutorials in Physics Sensemaking, Physics by Inquiry, and Next Generation Physical Science and Everyday Thinking) were compared with the Universal Design for Learning (UDL) framework. This framework originates in the education literature base and is composed of 3 guiding principles (1. Provide multiple means of representation, 2. provide multiple means of action and expression, and 3. provide multiple means for engagement) further described by 9 principles and 31 checkpoints. The UDL guidelines provide a framework for designing courses to be supportive of and accessible to all learners, taking into account variations among learners during curriculum development. Activities in these four curricula were analyzed for alignment between the in-class curricular elements and the UDL guidelines. Overall, all of the curricula aligned with two of the checkpoints: foster collaboration and community and support planning and strategy development. However, the curricula were unaligned with many of the checkpoints, specifically with regards to providing multiple means of engagement. Who we are prepared to teach indicates who we expect to participate in the physics community. We propose suggestions for modifications to existing curricula and for future curricula to better support all learners. We also argue that, if these research-based curricula do not meet federal legislative guidelines about accessibility for all students, the burden of creating an accessible environment and complying with these federal laws falls on the instructors, which could deter them from using the curricula. If we as a community want instructors to use high quality, research-based curricula, curriculum developers should prioritize supporting all learners.

Postdoc positions, intended to be advanced or continued research training in a field, are a part of the physics education research (PER) enterprise yet little is known about them. PER postdocs differ from their traditional physics counterparts in that they may have different education and research experiences. This study examined the types of postdoc positions available as well as what hiring is like. To determine the types of postdoc positions available, we used the advertisements posted in PERJobs blog. To learn more about hiring practices, interviews were conducted with both principal investigators (PIs) and current and former postdocs. Results show that many PER postdoc positions have been available since 2008, with many of these advertisements indicating that the PI will consider hiring someone with a physics or a PER or science, technology, engineering, or mathematics education background. The interviews indicate that there is no typical way for hiring PER postdocs. Some aspects that PIs consider include the background of the candidates and how well the candidates fit in with the group. Postdocs likewise consider whether they like the research group when accepting offers. For both, having ties to the PER community is important for hiring. Overall, PIs and postdocs have been satisfied. Postdocs who were originally in traditional physics have felt that being a PER postdoc has allowed them to transition to PER. In giving advice, both PIs and postdocs reiterate the importance of connecting to the community, evaluating the postdoc position as a stepping stone in the career trajectory, and holistically considering candidates.

[This paper is part of the Focused Collection on Astronomy Education Research.] This study investigates students’ perceived impacts regarding their participation in course-based undergraduate research experiences (CUREs) in astronomy. Each research experience adopted one or more projects from the Research Based Science Education for Undergraduates (RBSEU) curriculum, which teaches analysis of astronomical data coming from various national observatories. Participating students were enrolled in introductory astronomy courses at one of four universities using the curriculum. They were invited to respond to several instruments, including surveys ($N=199$), essays ($N=94$), and interviews ($N=19$). Each university implemented the curriculum differently with respect to content covered, length of instruction, and whether students’ research results were contributed to the astronomical community. We found that participation in all versions of the curriculum had the potential to significantly increase students’ perceived confidence participating in science. However, participation in experiences wherein results were contributed to the scientific community more often led to students’ nuanced perceptions of science processes, including increased understanding of the role of analysis and the utility of scientific communities and collaborations. We frame our study according to a pathway model under study by discipline-based education researchers of CUREs and explore our findings’ connections with psychological theories.

[This paper is part of the Focused Collection on Astronomy Education Research.] We examined teachers’ spatial-scientific reasoning and the alternative conceptions they held regarding Earth-space content. While participating in a professional development (PD) workshop, teachers engaged in an integrated mathematics and science project-based unit designed to foster spatial reasoning and improve lunar-related conceptual understanding. The Purdue Spatial Visualization Test-Rotation (PSVT-Rot) and the Lunar Phases Concept Inventory (LPCI) were used to assess understanding. We found the teachers held similar alternative conceptions as their students. Moreover, we discovered that teachers had limited understanding of the Earth-Moon-Sun scale, motions, and geometric configurations. To determine how teachers’ spatial-scientific confidence and ability translated to their classroom practice, we videotaped and analyzed instruction conducted by 6 teachers on an Earth-space lesson. Two teachers with the highest spatial-scientific confidence and ability had students achieving significant learning outcomes on both the PSVT-Rot and the LPCI.

[This paper is part of the Focused Collection on Astronomy Education Research.] This paper presents the first item response theory (IRT) analysis of the national data set on introductory, general education, college-level astronomy teaching using the Light and Spectroscopy Concept Inventory (LSCI). We used the difference between students’ pre- and postinstruction IRT-estimated abilities as a measure of learning gain. This analysis provides deeper insights than prior publications both into the LSCI as an instrument and into the effectiveness of teaching and learning in introductory astronomy courses. Our IRT analysis supports the classical test theory findings of prior studies using the LSCI with this population. In particular, we found that students in classes that used active learning strategies at least 25% of the time had average IRT-estimated learning gains that were approximately 1 logit larger than students in classes that spent less time on active learning strategies. We also found that instructors who want their classes to achieve an improvement in abilities of average $\mathrm{\Delta }\theta =1$ logit must spend at least 25% of class time on active learning strategies. However, our analysis also powerfully illustrates the lack of insight into student learning that is revealed by looking at a single measure of learning gain, such as average $\mathrm{\Delta }\theta$. Educators and researchers should also examine the distributions of students’ abilities pre- and postinstruction in order to understand how many students actually achieved an improvement in their abilities and whether or not a majority of students have moved to postabilities significantly greater than the national average.

[This paper is part of the Focused Collection on Astronomy Education Research.] The NASA/IPAC Teacher Archive Research Program (NITARP) partners small groups of educators with a research astronomer for a year-long authentic research project. This program aligns well with the characteristics of high-quality professional development (PD) programs and has worked with a total of 103 educators since 2005. In this paper, surveys were explored that were obtained from 74 different educators, at up to four waypoints during the course of 13 months, incorporating data from the class of 2010 through the class of 2017. This paper investigates how participating teachers describe their motivations for participating in NITARP as evidenced in these feedback forms. Analysis of self-reported data allows a mapping onto a continuum ranging from more inward focused to more outward focused; there is a shift from more inward-focused responses to more outward-focused responses. This insight into teacher motivations has implications for how the educators might be supported during their year with the program. This work provides a new way of parametrizing why educators participate in PD programs that require a considerable investment of time. NITARP, since it has many qualities of successful PD, serves as a model for similar PD programs in other STEM subjects. Likewise, the analysis method might also be useful to similarly evaluate other PD programs.

[This paper is part of the Focused Collection on Astronomy Education Research.] This qualitative study investigates astronomers’ and urban public middle school teachers’ beliefs about the characteristics of effective outreach partnerships in formal science classroom settings. Twelve astronomers and twelve science teachers participating in Baltimore Project ASTRO, a NSF grant-funded astronomer-educator partnership outreach program, were interviewed after participation using semistructured, in-depth interview techniques. Constant comparative analysis was used to analyze the interview transcripts. The findings suggest that astronomers and urban public middle school teachers believe that the characteristics of effective outreach partnerships center on the following three themes: partnership collaboration, astronomer and teacher characteristics, and astronomer and teacher dispositions. Regarding partnership collaboration, astronomers and teachers believed that establishing a relationship; maintaining communication; planning, preparing, and facilitating lessons collaboratively; and following up on lessons taught and modifying lessons for future implementation were vital characteristics for effective astronomer-educator partnerships. Concerning astronomer and teacher characteristics, effective astronomer partners were described as being willing to volunteer their time, willing to provide resources, and willing to purchase supplies. Effective educator partners were characterized as being responsible for managing the classroom and students, differentiating lessons and activities, and administering pre- and postastronomy assessments to students before and after their astronomer partner visits. In reference to astronomer and teacher dispositions, effective astronomers were prompt, able to relate to kids, exited, passionate about what they do, knowledgeable, tolerant, and easy going. Effective teachers were committed, motivated to have a volunteer astronomer in their classroom, passionate about teaching and about their students, and knowledgeable about the astronomy curriculum. The findings of this study help to provide a more comprehensive understanding of the characteristics of effective astronomer-educator partnerships in formal urban middle school science classrooms and help to provide essential guidance to all who are interested in designing and facilitating such science educational outreach programming.

[This paper is part of the Focused Collection on Astronomy Education Research.] This paper presents the cases of two doctoral students in observational astrophysics whose circumstances and experiences led them on a career trajectory out of academic research. In this article, I employ a sociocultural lens that provides insight into the dynamics of students’ experiences in astrophysics, which can in turn enrich our understandings of the lack of women in physics. I documented ethnographically two doctoral students’ experiences in a physics department at a large research university in Canada. In turn, I employed an analytic framework of figured worlds, cultural models, and identity trajectories to understand the challenges these two doctoral students faced. I use data drawn from observational field notes, interviews, and participants’ photo-narrative journals to explore the dominant cultural models of astrophysicist that were reproduced in their doctoral program. The analysis shows cultural models for recognizable astrophysicists in this doctoral program often did not fit neatly with these students’ experiences, and at times interfered with their trajectories into astrophysics careers. Additionally, results indicate that a prevailing discourse of gender neutrality and the rejection of normative femininity in astrophysics both afforded and constrained participants’ opportunities for recognition as insiders to astrophysics. However, the two participants repositioned themselves on alternative trajectories into physics teaching outside of academia, which had positive consequences for their astrophysicist identities. This study provides insights into the experiences of doctoral astrophysics that figure students’ insider or outsider identities, and the role that gender plays in the shaping of those identity and career trajectories.

[This paper is part of the Focused Collection on Astronomy Education Research.] Several decades of research have contributed to our understanding of students’ reasoning about astronomical phenomena. Some authors have pointed out the difficulty in reading and interpreting images used in school textbooks as factors that may justify the persistence of misconceptions. However, only a few studies have investigated to what extent usual textbook images influence students’ understanding of such phenomena. This study examines this issue, exploring 13–14 year old students’ explanations, drawings, and conceptions about three familiar phenomena: change of seasons, Moon phases, and solar or lunar eclipses. The research questions that guided the study were (RQ1) how are students’ explanations and visual representations about familiar astronomical phenomena affected by different image-support conditions? (RQ2) How are students’ conceptions about familiar astronomical phenomena affected by different image-support conditions? (RQ3) Which features of the used images most affected the students’ visual representations and explanations of familiar astronomical phenomena? To answer our research questions, we designed three instructional contexts under increasing support conditions: textbook images and text, teaching booklets with specially designed images and text, only text. To analyze students’ drawings, we used exploratory factor analysis to deconstruct drawings into their most salient elements. To analyze students’ explanations, we adopted a constant comparison method identifying different levels of increasing knowledge. To investigate students’ conceptions, we used a mixed multiple-choice and true false baseline questionnaire. For RQ1, results show that the specially designed images condition was effective in helping students producing informed drawings in comparison to text-only condition for all phenomena, and more effective than textbook images condition when one considers seasonal change drawings. Concerning RQ2, the specially designed images condition was the most effective for all phenomena. Concerning RQ3, prevalent elements of astronomy images that affected students’ explanations and visual representations were Earth’s elliptical orbit; the position of the Sun with respect to the Moon orbit; and Sun, Moon, and Earth alignment. Our findings confirm concerns about textbook astronomy images, whose features may interfere with the identification of the relevant factors underlying the phenomena. Moreover, findings of this study suggest that affordances of the specially designed images may play an essential role in scaffolding meaningful understanding of the targeted phenomena. Implications for teaching through and learning from visual representations in astronomy education are briefly discussed.

[This paper is part of the Focused Collection on Astronomy Education Research.] As part of a larger project studying undergraduate students’ understanding of cosmology, we explored students’ ideas about the curvature of the Universe. We investigated preinstruction ideas held by introductory astronomy (ASTRO 101) students at three participating universities and postinstruction ideas at one. Through thematic analysis of responses to questions on three survey forms and preinstruction interviews, we found that prior to instruction a significant fraction of students said the Universe is round. Students’ reasoning for this included that the Universe contains round objects, therefore it must also be round, or an incorrect idea that the big bang theory describes an explosion from a central point. We also found that a majority of students think that astronomers use the term curvature to describe properties, such as dimensions, angles, or size, of the Universe or objects in the Universe, or that astronomers use the term curvature to describe the bending of space due to gravity. Students are skeptical that the curvature of the Universe can be measured, to a greater or lesser degree depending on question framing. Postinstruction responses to a multiple-choice exam question and interviews at one university indicate that students are more likely to correctly respond that the Universe as a whole is not curved postinstruction, though the idea that the Universe is round still persists for some students. While we see no evidence that priming with an elliptical or rectangular map of the cosmic microwave background on a postinstruction exam affects responses, students do cite visualizations such as diagrams among the reasons for their responses in preinstruction surveys.

[This paper is part of the Focused Collection on Astronomy Education Research.] In this paper we discuss the design and development of a learning progression (LP) to describe and interpret students’ understanding about stellar structure and evolution (SSE). The LP is built upon three content dimensions: hydrostatic equilibrium; composition and aggregation state; functioning and evolution. The data to build up the levels of the hypothetical LP (LP1) came from a 45-minute, seven-question interview, with 33 high school students previously taught about the topic. The questions were adapted from an existing multiple-choice instrument. Data were analyzed using Minstrell’s “facets” approach. To assess the validity of LP1, we designed a twelve-hour teaching module featuring paper-and-pencil tasks and practical activities to estimate the stellar structure and evolution parameters. Twenty high school students were interviewed before and after the activities using the same interview protocol. Results informed a revision of LP1 (LP2) and, in parallel, of the module. The revised module included supplementary activities corresponding to changes made to LP1. We then assessed LP2 with 30 high school students through the same interview, submitted before and after the teaching intervention. A final version of the LP (LP3) was then developed drawing on students’ emerging reasoning strategies. This paper contributes to research in science education by providing an example of the iterative development of the instruction required to support the student thinking that LPs’ levels describe. Concerning astronomy education research, our findings can inform suitable instructional activities more responsive to students’ reasoning strategies about stellar structure and evolution.

[This paper is part of the Focused Collection on Astronomy Education Research.] We report on a study of almost 13 000 undergraduate students enrolled in introductory astronomy courses at the University of Arizona. From 1989 to 2016, students completed a basic science knowledge, beliefs, and attitudes survey. From 2014 to 2016, a subset of the sample completed an additional survey that probed their overall interest in science and astronomy, where they reported getting information about science, and their judgment of those sources. Our sample of mostly nonscience major students outperformed the general public on basic science knowledge questions during the time of the study. Additionally, there was very little change over the range of time of the study in students’ basic science knowledge whose scores averaged around 79% correct over the 27 years. Students’ self-reported demographic information and beliefs and attitudes in science and technology accounted for only 11% of the variance in their science knowledge scores and there was no systematic pattern between where students reported getting their information about science and their basic science knowledge. Despite this, there was a relationship between how students rated the reliability of sources and their science knowledge. Our findings support that introductory astronomy courses are opportunities to improve students’ attitudes towards science and ability to evaluate scientific information. Although this group of students’ basic science knowledge and attitudes remained relatively unchanged over 27 years there was a measurable relationship between students’ beliefs and attitudes, interest, science knowledge, and information literacy.

[This paper is part of the Focused Collection on Astronomy Education Research.] Students taking a second astronomy course for nonscientists were asked to reflect on the nature of scientific inquiry three times during the first half of the semester. First, they were assigned a short paper in which they were asked to argue for or against the thesis that observers of the night sky in ancient civilizations were scientists, and nearly all of the students argued that yes, they were. Second, they were asked to write a second paper in which they argued the opposite of their first thesis. Lastly, they were asked, on a midterm exam, to write an essay about whether the Ancient Greek astronomers were good scientists. In writing these papers and essays, many students included definitions of science or scientists, and these definitions progressed from paper to paper. In addition, the evidence students used to argue that ancient observers were scientists changed, with students becoming more expertlike.

[This paper is part of the Focused Collection on Astronomy Education Research.] An increased interest in using video games in educational contexts has led to many innovations in both formal and informal environments. Educational researchers, instructional designers, and educators have sought and developed ways to incorporate video games or gamelike elements into a curriculum. We present a study of a web-based college-level introductory astronomy class in which content is delivered entirely in a digital world setting through nonplayer characters and built-in minigames. Based on prior research on the potential benefits of video games for use in education, we expect that topics covered in the minigames, the most gamified aspects of the course, should be particularly effective. Using data collected through pre- and post-testing of two sections of ASTRO 001V (the designation given in Penn State’s course catalog) with the Test Of Astronomy Standards, we focused our analysis on six questions that pertain directly to minigame topics. We found that two of these questions showed encouraging gains, while the other four demonstrated that students continued to hold on to common alternate conceptions within those topic areas. This finding suggests that more work is required to understand how to improve the games in ways that will further support student astronomy learning.

[This paper is part of the Focused Collection on Astronomy Education Research.] Multiple studies show that spatial thinking skills contribute to students’ performance in science, technology, engineering, and mathematics disciplines. The study of astronomy is no different with the understanding of many astronomical phenomena requiring spatial thinking skills. This paper describes traditional and contemporary approaches to characterizing and measuring spatial thinking skills and suggests how they inform research in astronomy education. It summarizes previous literature in astronomy education research and categorizes the research approaches of astronomy education peer-reviewed journal articles and conference proceedings that explicitly consider the role of spatial thinking. Additionally, it recommends directions and curricular approaches for astronomy education research informed by current research in spatial thinking.