Nobody's Watching: Proctoring in Online Learning

There is no single best way to handle proctoring for digital courses, as this community college system pilot discovered.

  • By Dian Schaffhauser
  • 07/26/17

Using answers provided beforehand to pass online tests; checking a cell phone display to look up a formula in a closed-book exam; conferring with an unseen person in the room for the right responses. Cheating in an online course seems like it could be a simple matter for a motivated student.

That's why, as colleges and universities grow their online courses and programs, one aspect they need to nail down is how to ensure testing oversight traditionally handled by an impartial person sitting in the same room as the test-taker.

This is hardly a new challenge for schools. The Higher Education Opportunity Act of 2008 lays out the rule: An institution offering "distance education" needs to have processes in place for verifying student identity to ensure that the student who registers for a class is the "same student who participates in and completes the program and receives the academic credit."

Yet, short of requiring students to come on campus to take their tests or sending out human proctors to every nook and cranny in the region, how does a school balance a desire to expand access to online offerings while still ensuring academic integrity? That was the challenge for the California Community Colleges Online Education Initiative (CCC OEI).

Jory Hadsell, executive director of the CCC OEI, spoke at this year's OLC (Online Learning Consortium) Innovate conference and shared the solution: a combination of the use of an automated proctoring application and the creation of a network of colleges across the state that would provide no-cost proctoring on their campuses for students attending any of the member schools. Here's how CCC has succeeded.

The Background

The CCC has 113 independently accredited colleges, each with its own academic senate, curriculum and fiscal authority. There isn't necessarily course alignment among the schools, said Hadsell — and, by the way, the colleges "are set up to compete with one another for enrollments."

Twenty-four members of the CCC form the "pilot" colleges in the OEI, participating in a project to increase higher education degree attainment in the state while maximizing economies of scale by the sheer number of participants. As a body, they've adopted online course design standards, chosen Instructure Canvas as a common course management system and dipped into the use of open education resources. Those are fairly common practices among community colleges nowadays. But the "marquis" element of the OEI is an online course exchange.

Launched in 2016, the exchange provides the technical infrastructure to allow cross-enrollment among schools. "If a student needs a course that they can't get at their own college, there's an open seat at one of the other colleges," Hadsell explained. "In a really seamless way and with a couple of clicks they can enroll at that 'teaching college' and have all their units transfer back to their college." Getting that off the ground, he noted, "has been a heavy lift for us."

However, if students were going to able to obtain their credits online from any participating school, the one accepting those credits needed to know they were honestly earned. Likewise, the colleges were individually struggling to meet the regulatory requirements specifically related to student authentication.

Involve Faculty from the Beginning

OEI put together an online proctoring working group with "lots of faculty representation," said Hadsell, which "paid off in the long run." Other participants included people from testing centers and learning centers.

The first job was to evaluate options. An obvious one: using a proctoring service where somebody would watch the test-taker in real time but via webcam. Here's where the faculty input was important: They didn't care for that kind of solution. "We talked a lot about the psychological impact of entering into a live proctoring session and what it meant if there was a human being on the other end, what it meant if you were asked to do certain things that made you feel like you were a criminal even before you'd taken a test," said Hadsell.

The faculty reps also didn't want to delegate the job of figuring out whether a student was cheating or not. They really wanted that decision to rest with them. "Given the potential severity of a cheating allegation and the whole discipline process that has to kick in, they really wanted to be involved from the get-go, from the point of identifying something that looks suspicious to them, to being the one to interact with that student rather than having a proctor stop an exam or relying on someone saying what they saw," Hadsell recalled. "Having concrete proof was important to them."

Since the consensus seemed to be that faculty would want to review any allegation of cheating anyway, the working group decided to bring that capability in from the beginning. Besides, a software-based approach came across as more scalable, Hadsell added. "We have 1.3 million students. Trying to scale something with a 'human powered' solution wasn't necessarily the right fit." However, he emphasized, "We would have worked with that if that's the way the faculty had gone."

Try Proctoring Without the Proctor

Proctorio, the proctoring solution eventually recommended by the working group, is a web service that can be deployed through Canvas and installed by students with one click. That installation sets up a practice exam so students can try out the setup before tackling the real McCoy. Then, when they enter the actual exam, they acknowledge the rules and are presented with a screen asking them to hold up their ID to their webcam, which is recorded along with an image of their face. The webcam continues recording the exam and notes "abnormalities" — computer-based and technical (a browser resize, copy and paste activity or the number of websites visited during the exam) or environmental (odd movements, somebody looking away from the screen or voices in the room).

As the testing is done, the results are posted to a Proctorio "gradebook," an instructor dashboard with color coding and a score between 1 and 100 indicating the level of suspicion for each student as well as an icon that highlights possible exam participant collusion. The user can set the behavior analysis at three levels of intensity — "recommended," "lenient" or "moderate" — and the score will automatically change. If an exam experience warrants additional exploration, the instructor can go to a console that also uses color coding to guide the viewer to the places within the exam recording where irregularities were captured for review.

Stipulate a Webcam — Or Not

Of broad appeal to the faculty members was the amount of "variability" in Proctorio's software settings, particularly related to browser lock-down options, the ability to allow students to use class materials and even not requiring a webcam.

Affordability is always a consideration with CCC, which as an open-access institution has to follow strict rules about the material fees that can be charged to students. If a webcam is required for online exams, the schools have begun adding that stipulation to the course description so that students are forewarned. On top of that, Hadsell pointed out, webcams can be purchased for under $10 on Amazon — not much more than a couple of black eyes at Starbucks.

In extreme situations where that's not an option, the instructor could also reconsider how to configure the software. "There are a lot of powerful features that really live in the metadata that allow you to leverage the power of this," explained Hadsell. For example, the dashboard shows number of attempts on the quiz and details such as the average amount of time the test took. If the average is 42 minutes and a student completes it in six minutes, he suggested, "there might be a problem there worth digging into."

Go Multi-Modal

Proctorio satisfied a certain need for some faculty, but not all. For example, in the credit acceptance agreements that CCC maintains with the University of California and California State University systems, a few of the disciplines mandate face-to-face proctoring; they won't accept course credits unless that is in place.

The OEI working group examined the possibility of creating a network of reciprocal proctoring centers where students wouldn't have to pay a fee; the colleges would accept each other's students in their testing centers. Participating schools sign a two-page memo of understanding to join. Alongside setting that network up, OEI has introduced the idea of using Proctorio in the testing center to create a seamless student experience while reducing the burden on staff.

So far so good. From a college perspective, said Hadsell, most of the institutions "are happy to have an affordable software-based solution that has been vetted for accessibility, information security and student privacy concerns."

Remember: It's About Educating Students

No matter what kind of proctoring solution is put into place, Hadsell said he believes that as the semester goes on, there's less need for it. Why? Because students learn that even if nobody's watching, they're still being checked on, which changes their behavior. "We had a lot of conversations in our selection committee around what we were trying to do. Are we trying to deter incidences of cheating? Are we trying to inform students about what cheating is? Are we trying to catch them? It's probably all three."

6 Lessons Learned

Faculty buy-in is all-important. Most instructors have cobbled together their own proctoring practices. So getting them on board only happens when they've heard "from their peers that the workflow is not overly burdensome [and] that [they] don't have to go back and watch 46 hours of video after offering a quiz," said Jory Hadsell, executive director of the California Community Colleges Online Education Initiative (CCC OEI).

Find ways to use the proctoring for more than just high-stakes tests. "Validating student identity shouldn't be a one or two times a semester thing," noted Hadsell. "It should be something that's happening on an ongoing basis — for weekly quizzes, for homework."

Create sample language about the need for a webcam. The idea is that faculty can insert that into their syllabi and into the course description that students read before registering. "For us that generally satisfies the requirement that students knew before they took the class that this was something that they would need," said Hadsell.

Evaluate for low impact.In the OEI online proctoring working group's early conversations, the topic of dial-up connectivity surfaced. "Several of us snickered. Who's still on dial-up?" Hadsell admitted. But California, like any state, has remote regions where dial-up may be the only way students gain access to lessons. For that reason, the online solution needed to have low bandwidth requirements.

Don't rush into licensing. A per-exam model wouldn't work for CCC, observed Hadsell. "We wanted to make sure that this could have unlimited use for students who were using it in an unlimited number of courses for an unlimited number of exams," he explained. As a result, the deal the system negotiated with Proctorio calls for a one-time annual license cost per FTE student.

Keep up the momentum. OEI does webinars, on-demand videos, breakout sessions and dedicated events to train faculty on the use of the alternative proctoring systems. It has also created testing center guidelines to make sure rules are consistent.

Featured