Who Benefits? Positive Learner Outcomes from Behavioral Analytics of Online Lecture Video Viewing Using ClassTranscribe

Abstract

Lecture material of a sophomore large-enrollment (N=271) system programming 15-week class was delivered solely online using a new video-based web platform. The platform provided accurate accessible transcriptions and captioning plus a custom text-searchable interface to rapidly find relevant video moments from the entire course. The system logged student searching and viewing behaviors as fine-grained web browser interaction events including full-screen-switching, loss-of-focus, incremental searching events, and continued-video-watching events with the latter at 15-second granularity. Student learning behaviors and findings from three research questions are presented using individual-level performance and interaction data. Firstly, we report on learning outcomes from alternative learning paths that arise from the course's application of Universal Design for Learning principles. Secondly, final exam performance was equal or better to prior semesters that utilized traditional in-person live lectures. Thirdly, learning outcomes of low and high performing students were analyzed independently by grouping students into four quartiles based on their non-final-exam course performance of programming assignments and quizzes. We introduce and justify an empirically-defined qualification threshold for sufficient video minutes viewed for each group. In all quartiles, students who watched an above-threshold of video minutes improved their in-group final exam performance (ranging from +6 % to +14%) with the largest gain for the lowest-performing quartile. The improvement was similar in magnitude for all groups when expressed as a fraction of unrewarded final exam points. Overall, the study presents and evaluates how learner use of online video using ClassTranscribe predicts course performance and positive learning outcomes.