Researchers: MOOCs as Effective for Learning as Traditional Courses

• By Dian Schaffhauser
• 09/24/14

Massive open online courses can be as effective to help students learn as face-to-face classes — and the learning outcomes will be equivalent whether or not students have advanced college degrees or something far less than that, whether they're academically prepared for the class or whether they perform well or poorly on a pretest.

Those results come out of a study performed by a team of researchers from MIT, Tsinghua University and Harvard. According to research published in the current issue of the International Review of Research in Open and Distance Learning, students learn equally no matter their level of education, preparation, and overall ability in the course.

The study grew out of a MOOC developed at MIT that originated as an on-campus course to help struggling students understand "classical mechanics." "Mechanics ReView," a "second course" in introductory Newtonian mechanics, is taken in January by students who have received less than a C in MIT's large-enrollment fall Mechanics course. The review course uses a flipped classroom, in which class time is dedicated to teaching advanced problem-solving skills. The edX platform was used to deliver the online segment of the course for the cohort used in this study.

In June 2013 that same course made its way to the edX platform, and the researchers used various cohorts taken from that MOOC to compare results to the students in the on-campus course. From the MOOC the researchers worked with results from 1,080 students who attempted at least half of questions in the course (out of the 17,000 people who initially signed up). Ninety-five percent of those students earned a certificate. Most of the participants who did less than half of the homework and quiz problems dropped out and didn't take the post-test, so their learning couldn't be measured.

From the on-campus course side, the researchers worked with results from the spring 2013 session, which had 47 students, of which 35 attempted more than half of the online problems. Data from that group of 35 was used for the study.

According to the researchers, those on-campus students had benefits the MOOC participants lacked: "four hours of instruction in which staff interacted with small groups of students each week, staff office hours, helpful fellow students, available physics tutors and the MIT library." While the online students had "lively" online discussions, they also were required to do about 30 percent more problems, including more problems in the online tests than the on-campus students did in their weekly in-class tests.

However, those MOOC students could consult resources before answering questions, whereas the on-campus students took closed-book, no-Internet tests.

The study compared results of a pre-test and post-test given to both sets of students for evidence of learning and analyzed homework and quiz questions that were used in both formats.

"In spite of the extra instruction that the on-campus students had," the researchers reported, there was "no evidence of positive, weekly relative improvement of our on-campus students compared with our online students."

The results: "Learning was the same for different cohorts selected on various criteria: level of education, preparation in math and physics, and overall ability in the course," wrote the researchers. They did find a "small positive correlation" between relative improvement and prior educational attainment. They also found that the MIT freshman in the on-campus class showed "considerably less" skill in their homework than the MOOC students.

The results of the study are posted here on the journal Web site.