Research: Learning Is No Spectator Sport

• By Dian Schaffhauser
• 09/16/15

MOOCs that rely primarily on people watching lectures may be missing an opportunity to help their students learn even more by adding interactive activities. Recent research into massive open online courses suggests that students do six times better in the course by "extra doing." On top of that, they're more likely to persist in the course.

Five researchers from Carnegie Mellon University (CMU) undertook a study in 2013 when their institution and the Georgia Institute of Technology collaborated to add elements of Carnegie Mellon's Open Learning Initiative (OLI) course, "Introduction to Psychology," into Tech's "Introduction to Psychology as a Science" MOOC. Taught through the Coursera platform, OLI materials were available as part of the larger course, along with lectures, quizzes and other activities.

The initial release of the Tech offering was designed as a 12-week introductory survey course. Each week the program focused on a major topic, such as memory, abnormal behavior, and sense and perception. Those were broken into three sub-topics supported by a 10- to 15-minute video lecture with slides along with assigned modules and learning outcomes from OLI. Modules included a variety of expository content such as text, examples, images and video clips and a large number of interactive activities, such as reading scenarios or studying images and answering questions. A quiz assessed students against the outcomes at the end of each week.

The course also used Coursera platform features such as discussion forums, writing assignments and quizzes (with questions drawn from the OLI item banks. At the end of the course students took a final exam with questions created by the instructor.

Nearly 28,000 people registered for the course; 9,075 also registered for the OLI materials, and only a tenth of those took the final exam. The researchers examined when students dropped out and what factors might contribute to that decision. While quiz participation decreased over time across the entire group, they found that the rate was "consistently higher" — about twice as high — for the students who signed on for both the MOOC and OLI usage than for MOOC students in general. The biggest drop in participation occurred between quiz one and two, when 43 percent of students dropped out overall. Half of the dropouts were students pursuing the MOOC only and 39 percent of the dropouts were doing both the MOOC and OLI work.

When it came to the final exam, MOOC-only students (about 215 by this time) had an average score of nearly 57 percent; MOOC+OLI students (about 939) averaged almost 66 percent. The researchers found a causal impact from all course features, but the most influential one came from "doing" activities — more than six times the impact of watching video or reading text.

"Do students learn more with OLI? The answer is a clear and resounding 'yes,'" said Ken Koedinger, professor of human-computer interaction and psychology and first name on the research, in a press release. "With OLI, students get immediate feedback. If they do not master a concept, they have to go back to re-watch or re-read and then demonstrate they have learned before they are able to move on."

The paper, "Learning Is Not a Spectator Sport: Doing Is Better than Watching for Learning from a MOOC," is openly available on the ACM Digital Library site.

The Bill and Melinda Gates Foundation funded the courses' development and delivery. CMU's LearnLab, funded by the National Science Foundation, supported data storage and analytics.