(Software) Code of Honor

When USC's students began bringing computers to class, its law school turned to technology and rigid process to maintain academic integrity in the testing room.

• By Dian Schaffhauser
• 09/19/12

Cheating is in the news again--this time at Harvard, where about 125 students are being investigated for collaborating on an open book/open Internet exam for a government class that took place in May. According to the university, exam protocol issued to the students explicitly stated that, "Students may not discuss the exam with others..." Yet, the instructor for that course, according to reporting by student newspaper The Harvard Crimson, found amazing similarities among many of the tests, such as use of the same arcane examples and--in two students' exams--the same typo: a space in the number 22,500.

To students who exist in a world of free content, Wikipedia, and peer-to-peer file sharing, the concept of academic integrity may seem a quaint notion: Isn't everything ultimately sourced from something else? So, to emphasize its importance, one type of test management technology has surfaced in law schools all over the country from ExamSoft that could readily be used in other academic disciplines as well to maintain student honesty in the testing process. Use of the platform is also helping the students exposed to it to prepare for the high-stakes testing to come during their careers.

An Early Adopter
Jane Chang Bright began her job in 1996 as assistant director of student computing services for the University of Southern California Gould School of Law. At that point, the steps in administering final exams involved handing out sheets of printed questions and monitoring the room while students wrote answers in their blue books. The idea that students would collaborate on answers under the watchful eye of the proctor would have been laughable.

About a year after Bright started her job with the university, the law school was ready to allow students to integrate the use of their computers into the exam process. After all, students could edit their responses while they were taking the test, and the results would presumably be more readable. In that initial semester only half of the students took advantage of the use of a computer. "Not everybody had a laptop," she recalled. But that changed quickly. Now, although the law school has no laptop requirement, 100 percent of students have laptops and other computing devices, and most use those computers in the testing room.

To maintain the control it was accustomed to, the school adopted a new program called Xmn8r (pronounced, "Examinator") and since renamed SofTest from ExamSoft, which enabled test-takers to type their responses to questions in an interface, return to questions for revision, and use basic cut/copy/paste functions in order to better organize their answers before submitting them at the end of the test.

A "driving force" for USC's law school to use SofTest was because it was and still is the same software used by the State Bar of California in delivering the written portions of that set of tests. According to ExamSoft, 42 state bars use the company's products for their exams. The company estimates that 150 law schools with 120,000-plus students in the United States and Canada also use the program.

Turning a Computer into a Typewriter
Bright described what ExamSoft's software does as turning a computer into a typewriter. "It's no different," she insisted. Students can't get to their notes or the Internet. They can't open up PDFs. An exam log captures every keystroke. Should they try to turn off a computer during the exam in order to attempt to get around the lockdown, a lockout feature prevents them from getting back in--unless an exam administrator enters an unlock code.

In many ways, the testing experience with SofTest at the law school isn't much different from what it was pre-computer. In fact, in some ways the process is even more arduous because it adds extra deadlines to the student's calendar.

First, the law students need to download and register SofTest to their Mac or PC--usually about three weeks before the actual final is taken. If they miss that deadline, they also miss the opportunity to download their exam files, required if they expect to use their laptop for their test.

They have another deadline for downloading the exam files they'll be using to take the test with. The download process generates an icon on the student's desktop. The student launches that icon, which shows the available exams in a dropdown list. The student selects the exam, which downloads the exam files to their computer. Said Bright, "If they don't download exam files in time, they'll have to handwrite [their answers]."

It's common, Bright noted, for one or two students each semester to forget one or another deadline. They don't tend to do it more than once.

Also, nobody's allowed to take the test until they've handed in an online course evaluation before checking in to take the final. Those who don't complete the evaluation are given a paper copy of it, which must be filled out and handed to the proctor before they're allowed into the testing room.

When it's finally time for students to take their finals, they're actually handed a copy of the questions on paper. They're provided with an exam password and told to click "Begin" when they're ready to start the test.

Students are given a one-minute notice when the exam has started, to let them know it's begun and another alert two minutes before the test ends. Basic text formatting is available, and the institution can decide whether or not to allow the spellcheck function. Every minute, SofTest takes a snapshot of the file; every four minutes, it performs a backup.

When the student is done with the exam, he or she pushes an "Exit and save" button, confirms that it's time to close the test, and the computer returns to normal operation. At that point, SofTest uploads the answer file to ExamSoft, where it can be picked up by the institution for grading.

Interestingly, not all students choose to use SofTest. Opting out is particularly prevalent among international students participating in the school's Master of Laws (LL.M.) program, which makes up nearly a quarter of the 800 students in the law school. "A good number of them choose to handwrite their exams," Bright noted. But opt-outs also include one or two students pursuing their standard law degrees. Those individuals take their finals in a separate test "writing room."

An Exam Lifecycle
ExamSoft has bundled its testing modules for exam design and delivery, scoring and analysis, and administrative services into a suite it calls Flex-Site, which uses an online/offline model. It's online for the institution as a hosted service, and offline for students during a test.

USC's law school has used the exam design module only a little. SofTeach allows exam questions to be entered to avoid the paper step, but it's not practical for the law school, said Bright. "A lot of our professors don't get their exams in to be copied until the day of [or just before] the test," she noted.

One exception to that is an exam developed by the legal research faculty, the law librarians. In that case, Bright enters the exam questions into the program. But even then, the school discovered the first time it tried a digital form of the test, the students wanted to have a paper copy in order to mark up the questions. "We got so many complaints the first time the student weren't handed a hard copy of the exam, now the exam questions are not only programmed into SofTest, but they also receive a hard copy," she pointed out. "We're back to where we were before."

One ExamSoft product that has garnered more use at the school is SoftScore, an electronic scoring program. About a year ago the school began using SoftScore for its bubble sheet features in multiple-choice tests. This replicates the grading activity performed by Scantron machines. In the past, the process of having multiple-choice question-based tests scored at USC was for somebody in the law school to manually fill out a key with every answer, collect the test sheets, walk them across campus to the testing bureau, have the tests checked, return and pick them up when they're done being scored, and turn them over to the professor with the results. Whenever a faculty member decided a test needed rescoring, that required filling out a new key and repeating the process. Turnaround took at least a couple of days.

Using SoftScore, Bright can have the tests graded within half an hour of the exam files being uploaded. Rescoring takes a mere five minutes. The program generates a PDF or Excel file of results that's sent off to the instructor.

Stress Reduction for Students and Test Administrators
It's been so long since a cheating incident surfaced during testing at the law school, Bright said she has trouble remembering the details. "It happened maybe five years ago, where some first-year law students alleged another student was going into the bathroom and had stashed notes." She never found out whether the dishonesty really happened. However, it points to an important aspect of test administration: Besides establishing an expectation of academic integrity, students participating in the system have to believe that it's hard to cheat and that others won't get away with cheating if they try.

But at USC, as Bright said she sees it, there's an additional aspect to the use of the testing administration software that puts a positive spin on the controls and process in place.

She has worked in Washington as a contract proctor for that state's bar exam, and she described the endeavor as a huge room with easily 800 people, all of them outfitted with computers "and with no technical support."

The experience of taking the bar exam is "extremely stressful," Bright said. But some users make it even worse for themselves. The two most stressed out groups of people at the bar exam are those who are using a computer that's new to them, "because they're unfamiliar with the keyboard, with the mouse," and those who have never seen SofTest, "because their law school did not use it." In the scheme of all the hard work that students have gone through to get to this point in their fledgling career, those items may seem minor, she added, "but they still make a difference. I personally feel [that] a law school is doing their students and future grads a huge disservice by not using SofTest."