Comparing the effectiveness of online versus live lecture demonstrations

Nearly every introductory physics or chemistry course includes live lecture demonstrations, which can range from simple illustrations of a pendulum to elaborate productions with specialized apparatus and highly trained demonstrators. Students and instructors often consider “demos” to be among the highlights of these classes. Yet, in some situations demos may be cumbersome, inaccessible, or otherwise unavailable, and online video demos could offer a convenient alternative. We compared the effectiveness of live demonstrations with online videos under controlled conditions in the first semester of an introductory physics (mechanics) course. Students were randomly assigned to view either a live or video version of two demos. The same instructor presented both versions of the demo using an identical script, keeping the same time on task across both conditions, but with small differences in presentation appropriate to the medium. Compared with the students who saw the live demos, the students who watched the online videos learned more, and their self-reported enjoyment was just as high. We discuss reasons why videos helped students to learn more, including that they are more likely to make correct observations from the video. These results suggest that videos could provide students with an equally effective learning experience when live demos are unavailable. Indeed, even when live demonstrations are available, it may be beneficial to supplement them with online presentations.


I. INTRODUCTION
Live science demonstrations have a distinguished history: in the 19th century, huge crowds gathered to watch Michael Faraday illustrate new discoveries in physics and chemistry [1]. Today, instructors use demos to liven up "boring lectures" [2,3] and to help students develop and remember an intuitive, conceptual understanding of the world [2,4,5]. Successful demos can highlight-and ultimately resolve-students' misconceptions about the physical world. Demos illustrate physical principles that can otherwise seem abstract and disconnected from the real world [3], and can increase students' interest in physics [6].
Many institutions have invested considerable resources in materials, equipment, and staff to perform in-class demos. There is a canon of physics and chemistry demos that are used in introductory classes, almost as a course of habit. However, research has also shown that demonstrations are not always effective in promoting student learning and they can even lead to increased confusion [7][8][9][10]. In many cases, after watching a demonstration, students' understanding of what took place is not what the instructor intended, and these faulty observations may serve to reinforce students' misconceptions [11][12][13]. Miller et al. showed that conceptual learning is contingent on students making a correct observation of the demo in the first place [14]. As novices, students have difficulty extracting the intended physical information from a demo [15][16][17]. For instance, Roth et al. found that students' lack of understanding of the underlying concepts makes it difficult for them to separate a desired observation from background "noise" [15]. In that study, an instructor stood on a turntable while spinning a bicycle wheel to demonstrate the conservation of angular momentum. In the case when the instructor was not "supposed" to rotate at all, many students maintained that he did in fact rotate, because they observed slight rotations resulting from the demonstrator shifting position. For these students, these slight rotations were the real signal, instead of noise, and therefore the demonstration failed to illustrate the correct concept.
These limitations can be mitigated by presenting demos using best practices such as interactive lecture demonstrations (ILDs) [18]. ILDs guide students through an understanding of the underlying phenomenon by asking them first to predict the outcome of the demo and then discuss the results with one another. Crouch et al. found that, compared with students who simply observed the demonstration, students who predicted the outcome and discussed the results were better at recalling the correct outcome and correctly explaining the underlying physics [10].
There are other practical constraints to using live demonstrations. Maintaining a suitable number of demos can be prohibitively expensive: many involve specialized materials and equipment, and require considerable faculty and staff time for preparation and upkeep. Some demos require specialized facilities: high ceilings, large open spaces, custom power supplies, specific lighting conditions. Demos may fail during class, which can lead to embarrassment and stress for the instructor and confusion for the students. Many demos are time consuming to set up and, for practical purposes, can only be performed once. Demos can be difficult to see, especially in large lecture halls, and projecting a live video feed adds complexity and confusion because students do not know whether to focus on the physical demonstration or the screen.
Both the cognitive and practical limitations of demonstrations could be mitigated by presenting them using a high-quality online video instead of, or in concert with, a live demonstration. Animations and videos offer advantages in teaching dynamic phenomena, especially those that involve very rapid motion or abstract concepts [19,20]. Cognitive load theory [21,22] suggests that a well-edited video that removes distracting elements will reduce the cognitive load for the viewer and make it easier to process essential features of the demo. A video can slow down or pause dynamic phenomena and allow simultaneous verbal explanation, taking advantage of the dual-mode effect [23] and avoiding the split attention effect [24]. Indeed, as in the angular momentum study by Roth et al. [15], students often focus on irrelevant or distracting features because novices extract visual information based on perceptual salience [25]. A video can superimpose abstract vectors or other cues to guide students to the important concepts in a demonstration. Such cues are more clearly made in the online version, compared to the live version, wherein students' focus is more difficult to direct. Videos also facilitate the presentation of "contrasting cases," which are a powerful cognitive tool for abstraction and transfer [26][27][28]. In a live demonstration, an instructor could verbally describe a contrasting case but often cannot repeat the demonstration in real time to show the actual contrast.
Instructors might wonder: will students' learning or enjoyment be affected by the use of online demos? We seek to answer this question through a well-controlled study of two popular demonstrations from introductory physics (mechanics): (i) Shoot the Monkey and (ii) High Road, Low Road (a description of the live and online versions of these demonstrations can be found in the Appendix). We find that learning is higher and student enjoyment is at least as high with the online version of the lecture demos.

II. METHODS
This study took place during the first term of a year-long introductory physics course at Harvard University. This calculus-based course covers topics in mechanics at a level appropriate for life science and pre-medical students. For each demo, a four-minute-long script was written to showcase the demo and help illustrate the underlying physics. Each script made use of best practices such as prediction making [10,14,17] and contrasting cases [26][27][28]. For the live version of each demo, the instructor used gestures, blackboard illustrations, and physical props such as large wooden arrows to explain the underlying physics. For the online version, the same instructor recorded the identical narration and used superimposed graphics and animations to present the same concepts. Although identical scripts were used, there were inevitable small differences in presentation appropriate for each medium. The goal in each case was to provide the same conceptual physics content using the most natural idiom for each medium.
Students (N ¼ 110) were individually randomly assigned to two groups (A and B). For the Shoot the Monkey demo, group A experienced the video version of the demo, while group B experienced the live version (see Table I). For the High Road, Low Road demo, which took place several weeks later, group A experienced the live version of the demo while group B saw the video version. After each demo, students answered the open-ended question: "Write what you observed during the demonstration (please be detailed and include physics principles whenever possible)." Students also rated their level of agreement (on a 5-point scale) with the statement: "I really enjoyed this demonstration." This was followed by a 5-item multiple-choice test of learning on their conceptual understanding of the underlying physics. For instance, students were asked, "Let's reconsider the monkey gun demo. If the bullet is now fired twice as fast as before, where should you aim the gun so that you still hit the monkey? (a) Above the monkey, (b) slightly above the monkey, (c) directly at the monkey, (d) slightly below the monkey, (e) below the monkey." The survey questions and both tests of learning may be found in the Supplemental Material [29]; the questions were drawn from a set of conceptual clicker questions that have been used for many years in this course.
The study design featured several controls to ensure consistency and avoid bias: (i) As discussed above, both versions of each demo used the same instructor, script, and conceptual content. (ii) The time on task for both versions of the demo was kept precisely the same by having the "online" version projected to students in a separate lecture hall, controlled by a proctor who paused the video at predetermined times, such as when students were asked to make a prediction. (iii) In both conditions students were asked to put away all other materials that might serve as distractions (phones, computers, papers, etc.). (iv) Each student experienced both modes of demo in a "crossover" study design that controls for unobserved variation between students or groups. Figure 1 compares students' performance on the postdemo test of learning (left) and students' self-reported enjoyment (right) of the live vs video versions of the two demos. On the test of learning, students who viewed the video outperformed students in the live groups by 25%-30% (p < 0.01). This suggests that students learn more from the online versions of these demos than from the live versions. For the High Road, Low Road demo, the difference in student enjoyment between the two versions was not statistically significant (p ¼ 0.65). For the Shoot the Monkey demo, students in the online group enjoyed the demo more than students in the live group (p < 0.02). Notably, placing the demos online in video format does not appear to diminish students' enjoyment. We conducted oneon-one, structured interviews to validate how students interpret the survey statement "I really enjoyed this demonstration" (13 students total). These students provided a representative sample of the entire population as measured by their CLASS scores, FCI scores, and final course grades. These interviews revealed that students interpret this statement primarily as (i) how much they learned from the demonstration, (ii) how it helped them better understand the lecture, and (iii) how it illustrated physics as applied to the real world.

III. RESULTS AND DISCUSSION
Research has shown that conceptual learning from demos is contingent on students observing the demos correctly in the first place [14]. Thus, if online videos facilitate correct observation, students might learn more from this format. To test this hypothesis, we coded students' written descriptions of each demo for correct observation of all key phenomena. For the High Road, Low Road demo, 63% of the online group had correct observations compared to only 51% of the live group. For the Shoot the Monkey demo, 88% of the online group correctly observed the demo compared to 57% of the live group. For instance, when describing the live Shoot the Monkey demo, several students incorrectly claimed that the cannon was intentionally aimed slightly below the initial position of the monkey, and they explained that this was required because the ball travels a much longer path than the monkey does while it is falling.
As discussed in the introduction, videos have several features that help students make more correct observations. In this particular instance, the instructor in both versions (live and video) followed the same script and said, "Maybe you guessed we should aim below the monkey, but that would seem to shoot too low." In the live presentation, it was impossible to repeat the full demo to show this contrasting case, so the instructor used hand gestures and the blackboard instead. Online, a video of this case is shown in quick contrast to the case where the ball is aimed directly at the monkey. Moreover, the video is displayed in slow motion with synchronized narration so students can focus on the trajectory of the ball as it passes below the monkey. Although the same information was presented in both versions, the video has clear benefits in helping students avoid the incorrect observation that the cannon was intentionally aimed below the initial position of the monkey. Other learning benefits of contrasting cases, such as abstraction and transfer, are also likely enhanced by the video. Finally, we note that abstract concepts such as the horizontal component of a force vector are more readily presented on video compared with live demonstrations which must rely on verbal descriptions, blackboard illustrations, and physical cues such as wooden arrows. Thus, there are at least three factors-correct observation, clearer contrasting cases, and animated abstract conceptsthat could contribute to greater learning from online vs live demos.

IV. CONCLUSION
The literature shows that student learning from live demonstrations is limited, although these limitations can be mitigated with best practices. Financial and logistical complications add to the constraints of live demos, making it difficult for institutions without equipment and a dedicated staff to use demos effectively in lectures. Despite these constraints, instructors still use live demos: they clearly excite interest and students enjoy them. However, as we have shown with the two demos we studied, students can enjoy online video versions of the demos just as much as the live versions. Further research is needed to see if these observations generalize to all demos; there could be a subclass of demonstrations for which students will always enjoy the live version more than an online version. More importantly, students also appear to learn more from online vs live demos, as shown by our empirical results (tests of learning) and supported by cognitive principles (correct interpretation of demos and clearer contrasting cases).
Given these observations, as well as all the practical advantages of online demos, we encourage instructors to use them in lecture, especially for demos that are otherwise inaccessible or in learning environments (such as distance education courses or MOOCs) where live demonstrations are impossible. Even at schools with abundant resources, it may be beneficial to supplement live demonstrations with online presentations. More research is needed into optimizing the effectiveness of online demonstrations, and the growth of online demos is an opportunity to bring these valuable teaching resources to a much larger audience of students.

ACKNOWLEDGMENTS
The authors are pleased to acknowledge significant contributions from Eric Mazur, along with valuable discussions with Melissa Franklin and Anna Klales. The Harvard Science Demonstration team of Wolfgang Rueckner, Allen Crockett, Daniel Davis, and Daniel Rosenberg provided valuable assistance in preparing and presenting the live demonstrations and the filming of the online demos, as well as the content of the live lecture demonstrations in the Appendix. This work was supported by Harvard University through the Division of Science in the Faculty of Arts and Sciences.

APPENDIX
The following is a description of the lecture demonstrations used in this study, including a list of benefits for the online versions.

Description of Shoot the Monkey demo
In the Shoot the Monkey demonstration a cannon is aimed directly at an object suspended several meters away, at a height a few meters above the cannon. The moment the cannon is fired, the suspended object is released from rest. Since the gravitational acceleration is identical for the projectile and the falling object, the moment the horizontal positions of the projectile and falling object coincide, so do their vertical positions, and the two collide. The height above the ground where the two objects meet depends on the projectile's initial velocity and the horizontal distance between the cannon and the falling object. This is a demonstration of the independence of the horizontal and vertical components of the velocity of a projectile. Schematics and images of the setup are shown in Figs. 2 and 3, respectively.

Benefits of the online Shoot the Monkey demonstration
In the online demonstration, the following contrasting cases of the demo are shown in quick succession and in slow motion: (i) aiming above the monkey, (ii) aiming below the monkey, and (iii) aiming directly at the monkey (see Fig. 4). Since it can take several minutes to set up the demo each time the projectile is fired, the same contrasting cases are shown in the live demo using the blackboard to draw the trajectories. Similarly, a second set of contrasting cases with the ball shot at different speeds is shown in quick succession and in slow motion (see Fig. 5); again, the instructor in the live demo follows the same script using the blackboard to show the contrasting cases.
A third contrasting case is used to illustrate the effects of gravity along the horizontal and vertical directions; the trajectories of the projectile and target are shown for the cases when gravity is turned ON and OFF. The online demo shows this using animation, whereas the blackboard is used for the live version.

Description of High Road, Low Road demonstration
In the High Road, Low Road demonstration, two balls, starting with the same initial horizontal velocity, roll along two different paths: one takes a straight horizontal path (high road) while the other rolls down into a valley, follows a short straight horizontal stretch, and then back up again to the original height (low road). The horizontal distance traveled by each is the same, but the low road is a longer path. Crucially, the x component of velocity for the "low road ball" while traveling into the valley, is always larger than the x component for the ball on the high road, and therefore "wins the race." The two configurations used in this study are shown in Fig. 6.

Benefits of the online High Road, Low Road demonstration
The video demonstration allows the two contrasting cases shown in Fig. 6 to be shown in quick succession and in slow motion. As with the "Shoot the Monkey" demonstration, we also show a simplified simulation of the demonstration, where the normal force and weight of the ball are visually displayed throughout the motion, as shown in Fig. 7. (Rotation and friction have negligible effects on the motion and are therefore omitted.) When students see that the normal force has an x component along one path but not the other, it can help clarify why the low-road ball accelerates horizontally while the high-road ball moves at a constant speed. In the live version, the instructor follows the same script using the blackboard and wooden vector arrows.