MOOCs

Research Uncovers MOOC Cheating Strategy

• By Dian Schaffhauser
• 08/26/15

As universities experiment with granting credit hours or digital badges to MOOC students, they may want to implement simple strategies to prevent cheating. Yes, even when the class is free and the only gain may be a certificate of completion, courses that are massive, open and online face one of the same kinds of problems as traditional courses do.

Researchers from MIT and Harvard University have uncovered a new cheating scheme specific to MOOCs. As they explain in a working paper freely available online, some students are taking advantage of design features that allow for the creation of multiple accounts for a MOOC platform. They use some of the accounts to ferret out the right and wrong answers to quizzes; then they use one remaining account to submit only the correct answers.

The approach, said co-author Andrew Ho, a professor at Harvard's Graduate School of Education, is a "method of cheating that allows you to acquire a certification for a course in an hour, which is not possible through conventional cheating approaches. This is cheating of a different kind."

The outcome could make a serious dent on the reputation of MOOCs. "If learners in some online courses are circumventing the learning process and obtaining certification without going through the traditional routes of assessment and feedback, then the certification does not necessarily imply that they learned anything. This could seriously devalue MOOC certification," added co-author Isaac Chuang, a professor of electrical engineering and physics and senior associate dean of digital learning at MIT.

The researchers have come up with a name for this approach: "copying answers using multiple existences online" or CAMEO.

They uncovered the CAMEO technique while examining data from edX, the MOOC platform that MIT and Harvard both helped found. Certain users were answering assessment questions "faster than is humanly possible," said Chuang.

To uncover how much the approach was being used, the researchers created an algorithm to identify CAMEO users by using a series of filters. In the process the formula helps identify all possible pairs of users who provide correct answers "fairly quickly" — within five minutes and frequently only seconds — after submitting incorrect ones. An additional filter identifies all possible pairs of accounts in which one participant is signed up as having no interest in certification and the other is. Another filter seeks those pairs who share an IP address either for the given assessment or during any of their course-taking history. A final filter excludes all pairings who are part of a group with 10 accounts or more to eliminate people on shared routers in "classrooms or cafes" that could generate false positives.

With the algorithm in place, the faculty members tested data from 1.9 million course participants in 115 MOOCs offered by HarvardX and MITx from the fall of 2012 through the spring of 2015. They found 69 courses where users were found to have been employing the CAMEO approach and 1,237 certificates earned through this form of cheating. Also, among MOOC students who have earned 20 or more certificates, a quarter of the certificates appear to have been won through CAMEO, which could account for up to five percent of all certificates earned on edX.

CAMEO users are more likely, the report stated, to be "young, male, less education, and international." The countries with the highest CAMEO counts were Albania (12 percent), Indonesia (four percent) and Serbia (three percent). Within the United States, the rate was 0.4 percent of certificates earned, which, the authors suggested, may reflect the perceived differences in the value of certification among countries.

While the numbers may seem tiny in comparison, the ramifications are important, they noted. "This is not an isolated incident of copying on part of one assignment," Ho says. "This is the wholesale falsification of a certificate."

Preventing CAMEO may be as simple as mixing up the questions from one test-taker to the next or withholding answers until after all assignments are due, according to the report. But content providers will have to consider the trade-offs. For example, the report said, "If instructors withhold the 'show answer' option until after the problems are graded, this would constrain generally desirable asynchronous MOOC usage, and students will not have the rapid feedback touted as a pedagogical benefit of online learning environments." Also, randomizin questions are easier in some types of courses than others.

The researchers are ultimately hoping that course content creators will put some of the prevention strategies in place. "One of the most interesting lessons from the paper is that there are ways to mitigate cheating that are straightforward and implementable by the teams creating online course content," Chuang said. "We also expect platform improvements, such as virtual proctoring, to help reduce cheating."