Massive Open Online Courses | Feature
Massive: What Good is the M in MOOC?
Amherst College's recent and much publicized decision not to join nonprofit massive open online course (MOOC) provider edX draws a stark line between small classroom instruction and the online courses that sometimes enroll more than 100,000 students at a time.
Amherst's Chair of Neuroscience Stephen A. George led the faculty rejection of edX. It wasn't a rejection of online learning or open resources or the idea of making entire courses available for free online that they rejected, he said. "It was just the massive, synchronous MOOC that didn't seem to fit" with the school's mission and identity.
While few would quibble with the claim that in-person instruction benefits from smaller class sizes, a case can be made that online courses--and perhaps instruction as a whole--may be improved when massive numbers of students participate.
Student Interaction
It's not immediately apparent from the outside how a course with tens or even hundreds of thousands of students from around the world could create any kind of collaborative community conducive to cogenerative learning. But Michelle Rhee-Weise, education senior research fellow at Innosight Institute, explained that although the peer-to-peer learning looks very different from what we expect in a classroom, "we shouldn't be looking at these online courses as a replication of what's going" there.
"There are misconceptions about how MOOCs don't facilitate collaboration between students," Rhee-Weise explained. "In fact, what a lot of professors are finding is that students are answering questions for each other more so than they were in a classroom where you might have a little bit of shyness or trepidation in terms of participating in a more public atmosphere."
MOOC believers back that claim with numbers. Computer science professors Joe Warren and Scott Rixner are currently in the middle of teaching a second round of their Rice University MOOC, "An Introduction to Interactive Programming in Python." During a phone interview, they pointed to a recent question posted by a student to the course's forum. It was only 21-minutes old, they said, but had already received six responses from other students.
edX Director and MIT Professor Anant Agarwal echoed them. "In fact," he said, "in our spring course of 2012 the median response time to answer was 11 minutes."
Warren acknowledged that fast responses are not necessarily good or correct responses.
"Some of the answers," he said, "are good and some of the answers are bad, but I would point out that actually this is an incredibly important skill to learn outside of programming, that the ability to read and collect information and figure out who knows what they're talking about and who doesn't is actually probably more important than learning how to become a programmer."
Rhee-Weise suggested that students respond to one another's questions better than Warren's comment might suggest. "Actually," she said, "the students [in Agarwal's first edX course] were very much on track in terms of providing educated responses."
For those occasions when student responses don't provide the assistance a student requires, Rixner and Warren have deployed a helpdesk staffed by themselves and a dozen community TAs. So far, they said, they've been able to provide all the personal interaction students have requested across one-and-a-half terms of their Coursera-hosted course.
Economic Incentives
Moshe Vardi is a colleague of Warren and Rixner's in Rice's computer science department and the editor-in-chief at Communications of the ACM (CACM). Something of a MOOC skeptic, he wrote an editorial for the CAMC late last year that concluded, "If I had my wish, I would wave a wand and make MOOCs disappear, but I am afraid that we have let the genie out of the bottle."
While he was more willing to talk about the potential benefits of MOOCs during an interview with Campustechnology.com than the above quote may suggest, he said that the main benefit of massive courses is the economic "leverage of essentially using one professor teaching thousands of students."
"So now MOOCs give you a promise that you can suddenly change the productivity side of education," he said. "In the last 30 years I have never seen any topic in education that would suddenly get everybody's attention in such a way."
Vardi also pointed out, however, that such a vision of improved economics is largely illusory to date, as no one has figured out how to monetize it yet. "So far, MOOCs are open, which means they are free," he said. "Free is not a good business model."
George, who said that the Amherst faculty had "no particular quarrel with edX" and pointed out that they had decided not to join Coursera or Udacity, either, said he is also suspicious of the economic incentives of some people backing MOOCs.
He pointed to phrases uttered by MOOC evangelists such as "creative destruction" and comparisons of the movement to a "tsunami" sweeping higher education.
"There's a lot of money involved in higher education," George said, and he wondered if "they're going to try to remake it for their benefit. There's a question as to whether it is some kind of altruistic venture that really aims to provide worldwide education for free."
For their part, Warren and Rixner laughed when asked if they had come up with a way to monetize their MOOC.
"We are not making any money from doing this at this point and I think that's not our current objective. Our current objective is to understand the technology, see what we can do, see how we can deliver the best classes we can and, like Joe said, bring back some of these things to the campus," Rixner said. "I think that both of our classes have become significantly better here on campus because of this experience. So we're not looking for money right now."
"I like the fact that we have a highly rated class," Warren added. "My reward system is the fact that the students really love the class, so for me that's actually more valuable than bringing in some amount of money. Now, if at some point people are getting fabulously wealthy, maybe I'll revisit that, but at this point I like the fact that I have thousands of students that literally think I'm the best thing since sliced bread."
Agarwal said that the massiveness of MOOCs "creates one of the big efficiencies of online courses when a single professor with one TA, for example, can teach anywhere from 100 to 1,000 times as many students as they would teach in a normal class" and acknowledged that "data is very valuable." He also pointed out that edX is a nonprofit that makes all the data it's collecting "--in anonymized form--available for free to all our partner universities. And so all our universities get to see the data for free and to learn how students learn and do the analysis."
Massive Data
"We are gathering data on every event that a student submits to our system," Agarwal said. When does a student start "watching a video, when do they rewind it, what answers are they providing to assessment, how often they try different things, what time are they accessing it, what page of the book they're looking at? So we have a lot of this data that we're gathering. We can look at the data and we can try to come up with some patterns and so on to understand how students are learning."
Agarwal pointed to research by Physics Professor David Pritchard, also of MIT, who used his edX MOOC to look into how students used course materials as they performed different tasks. Pritchard's research showed, for example, that students working on homework spent more time watching videos and visiting the forum, while they were more likely to be found reading the online textbook while they were taking exams.
"We're also doing experiments. For example, in spring 2012 we did what is called A/B testing. You give some fraction of students one kind of experience, and you give a second fraction of students a different kind of experience. And then you see how they did on the course based on these different experiences, which would then tell you which experience, A or B, might be better to show students in the future," Agarwal said. "We're able to do these experiments and gather data from very very large numbers of students and we're able to do them over one offering of the course, which previously may have taken researchers 50 years or 100, which would make it impractical to get the results."
While acknowledging that MOOCs are gathering enormous amounts of data, Vardi and George questioned how useful it will prove to be.
Vardi compared it to looking for lost money under a streetlight not because you thought that's where it was likely to be, but because that's where you have enough light to see.
"The kind of data that they're gathering is typically people clicking on a radio button, true/false, or a multiple-choice question online and I think it is yet to be proven whether that kind of data is really going to inform the kind of education being performed at a place like Amherst," said George. "Furthermore, there is already an overwhelming amount of data showing how much better people learn in short segments followed by a question than just listening to an hour and a half lecture or something like that. So there's a lot already known and there's no indication, based on the kind of data that they seem to be trying to collect, that anything will be discovered. I can't say for sure that it won't, it's just that I don't see it from looking at how they're collecting data so far."
Massive Registration Leads to Massive Non-Completion
One aspect of online education that clearly does not benefit from massiveness is completion rates.
Katy Jordan, an education researcher and graduate student at The Open University, has published a graph on MOOC completion rates. According to information included with the graph, "completion rates can approach 20 percent, although most MOOCs have completion rates of less than 10 percent."
But do completion rates matter? Rixner and Warren pointed out that there's next to no cost to sign up for a MOOC out of curiosity and with no intention of ever doing any coursework.
Stephen Wan, a freshman at the University of Texas at Austin, has blogged about his experience taking MOOCs through edX and Coursera. He has taken three, though only finished one, which he indicated was intentional. In a phone interview, Wan said that when one of his friends was taking a MOOC on signal processing, he was "interested in learning how DSP chips, how signal processing chips worked, so I listened to the lectures on those" and then he left.
Wan said he met his personal learning goals, yet didn't complete the course.
Defining completion rates as the number of students who gain a certificate, Jordan said, means that "while they are not the most important thing about MOOCs--there are many other ways that participants can potentially benefit from MOOCs without necessarily gaining the certificate--it is not an issue that should be dismissed entirely."
"Completion rate obscures a different issue--that is, how many students set out to complete the course but did not succeed," Jordan said. "For example, in one of the MOOCs which I studied on Social Network Analysis, 2,417 students took the final exam and 1,410 got a certificate (either standard or with distinction), which would suggest that over 40 percent of the students who stuck with the course to the end did not achieve a certificate."
"Improving the ‘pass’ rate of a course then becomes a pedagogical issue," Jordan continued. "Given that MOOCs are open to all and are free from entry requirements--which is a fantastic move--professors may need to find different ways of teaching complex concepts to a lay audience, and I think this is a key issue for improving pass rates. I would also point out that this is ‘uncharted territory’ to an extent, and MOOC providers are also learning about what works at this scale."
Experimentation at Scale
Two words that came up over and over among massive course leaders and skeptics alike were experimentation and innovation.
Vardi said that he has his "doubts about the ultimate value of MOOCs in their current form," but he's "very optimistic what new and innovative things will come" from them and the conversation about pedagogy they have sparked. He even said he would support Rice in joining the edX Consortium "because this is happening and I don't think it makes sense for us to sit aside."
As he described different ways a MOOC might try to create persistent cohorts, as edX is currently doing, Agarwal cautioned that "all of this is a big experiment in progress and we have to understand that we don't have an answer. There's not an answer. We need to experiment with this technology and see how it works."
From Amherst, where faculty members turned a philosophical commitment to smallness into a huge voice in the conversation about massive online instruction and pedagogy by refusing to participate in edX's brand of experimentation, George talked about how his institution has been approached by organizations interested in helping them create something similar to MOOCs in its openness and pursuit of experimentation.
He turned again to the analogy of a tsunami that has the potential "to completely devastate higher education. And if that's true," he asked, "why not instead of trying to jump in and probably get dashed to pieces, move to higher ground, let that single wave go by, and then let's look at the landscape [of innovation] after that, and soon we can find what's useful, and then help the survivors."
