Blended Learning | Research
Interactive Online Learning Produces Learning Outcomes on Par with Traditional Teaching Methods
Hybrid teaching methods that combine interactive online learning with limited classroom teaching produce equivalent learning outcomes to traditional classroom teaching methods, according to new research released recently.
The report, Interactive Learning Online at Public Universities: Evidence from Randomized Trials, found that when students completed an introductory-level statistics course using an ILO system, along with one hour each week of classroom instruction, their scores on a standardized test of statistical literacy (CAOS) administered before and after the course showed equivalent gains to scores from students who completed the same course through traditional classroom teaching methods only.
In fact, the students in the hybrid course had slightly higher scores than those of classroom-only students, although the differences in scores were not statistically significant. However, the students in the hybrid course devoted 25 percent less time to coursework than their classroom counterparts did, a significant difference that researchers suggest may lead to opportunities for cost savings or greater learning outcomes.
The report's authors speculated that using hybrid teaching methods for some large introductory courses would enable institutions to increase enrollment without increasing the costs associated with classroom space. Consequently, it could increase access to education by making it more affordable for educational institutions to enroll more students and provide greater scheduling flexibility.
The study was conducted and funded by Ithaka S+R, the strategy and research arm of Ithaka, "a not-for-profit organization that helps the academic community use digital technologies to preserve the scholarly record and to advance research and teaching in sustainable ways."
It took place in the fall term of 2011 at six public university campuses in the United States: two at State University of New York (SUNY), two at the University of Maryland, and two at City University of New York (CUNY).
The 605 students who participated in the study represented diversity in race, age, scholastic achievement, education level, socioeconomic background, and other factors. They were randomly assigned to either the hybrid or traditional format course. At the end of the course, the study found "no statistically significant differences in learning outcomes between students in the traditional- and hybrid-format sections."
"We also calculated results separately for subgroups of students defined in terms of various characteristics, including race/ethnicity, gender, parental education, primary language spoken, CAOS pre-test score, hours worked for pay, and college GPA," researchers noted in the report. "We did not find any consistent evidence that the hybrid-format effect varied by any of these characteristics."
While there was no quantitative difference in the learning outcomes between the two groups, there was a qualitative difference in the students' experience. Perhaps surprisingly, the students in the hybrid format reported enjoying the course less than the students in the traditional format.
The interactive online learning (ILO) component of the hybrid statistics course was a prototype developed by at Carnegie Mellon University. According to the report, the researchers "found that the students gave the hybrid format a modestly lower overall rating than the one given by students taking the course in the traditional format (the rating was about 11 percent lower)."
One of the authors of the report, Matthew M. Chingos, a senior research consultant at Ithaka S+R and a fellow at the Brookings Institution's Brown Center on Educational Policy, said the prototype ILO course lacked the engaging features students may expect from an online course, which would explain the lower rating they gave it than the traditional course.
"If you go and look at the course, it's interactive and it has a lot of good features, but the aesthetics aren't that great," said Chingos. "It looks like it was designed in 1995. One person said to us it looks like it was designed by cognitive scientists, you know, no offense intended."
Chingos said he hopes ILO systems with better aesthetics and approaches to student engagement might be more successful at gaining student favor.
"If you could imagine a future version of a course like this, taking advantage of more addictive, interactive, exciting type features, getting students even more involved in it, the hope is that in the long run, this won't just produce the same outcomes in less time for less money, but will actually improve the quality of the educational experience," said Chingos.
According to Chingos, there were two reasons the researchers chose to use the hybrid teaching method rather than a straight ILO method. "In order to benefit from the rich feedback loops embedded in the ILO system, it's useful to have an instructor who can see how well students are doing and take advantage of that information in these weekly face-to-face meetings," said Chingos. "The other reason was that a lot of campuses aren't ready to go online only. They have students who are in residence or live nearby and are expecting to get some face-to-face time, so I think there are real political reasons, especially as you are making a transition to these new kinds of blended learning models. I think there are also important pedagogical reasons."
The study focused on an introductory level statistics course, and Chingos said he can't speculate on what the learning outcomes would be for higher level courses or courses in the humanities.
"I think the ILO type method has the most promise for classes where there's a right answer," said Chingos. "It's also very important whether it's introductory or later on, but for classes like math and statistics, where you're learning concrete things and then you're doing problems that have a right or wrong answer. It's hard to imagine it working for an upper-level Shakespeare seminar, where the concept of the class is really more discussion-oriented."
Nevertheless, Chingos said he believes the results of this study should help dispel fears about online learning hurting student outcomes.
"This method appears to have potential," said Chingos. "I view this research as a proof of concept that at least one instance of a high quality online course can produce equivalent outcomes and hopefully future versions that are higher quality will do even better."
The complete report can be accessed freely on Ithaka S+R's site.
