Electronic Grading: When the Tablet is Mightier than the Pen

• By Jeffrey L. Popyack, Nira Herrmann
• 12/30/02

Technological advances often seem to introduce as many obstacles as improvements. For instance, people seem to spend more time reading and writing electronic mail than they ever did with regular mail. Likewise, VCR use is sufficiently formidable to have inspired folk humor about how to program one.

And so it is with grading assignments that have been submitted electronically. Electronic submissions of student assignments certainly provide many advantages for the faculty member and graders. For instance, they are easier to manage and keep track of than their paper counterparts, particularly as the number of submissions increases. Submissions can be time-stamped automatically and archived, thus minimizing the potential for disputes over lateness and lost assignments and/or grades. Furthermore, archives can help resolve issues involving academic dishonesty and/or plagiarism. Finally, paperless transactions are especially useful when the assignments must be distributed to multiple locations, such as to teaching assistants, graders, and plagiarism detection software.

Drawbacks of Electronic Submission
Assignments submitted electronically are not without drawbacks, however. People are more familiar with manipulating paper documents and editorial markup, in particular, has been much simpler with a paper document. Consider, for instance, a student composition submitted as a text document where the grader wishes to cite a particular passage as requiring revision. On a paper document, the grader simply circles the offending passage, crosses out a few words, draws arrows or other editorial markings, and writes remarks directly on the paper. The grader's remarks (perhaps in red ink) stand out as distinct from the original submitted work, and context is understood from their location on the paper. The grader can work in close or cramped quarters, compare papers side-by-side, and easily flip back and forth between pages.

With a document in electronic form, the grader must decide how best to convey remarks while leaving the original submission intact. Users have tried a variety of approaches, but to date, they have all posed some drawback in practical use. One approach is to print a copy of the original document, then mark it up and return it. This, however, places the burden of printing on the grader (prohibitive in time and expense for large classes), and d'es not preserve an archival record of the grader's remarks.

Another approach is for the grader to type a separate document with remarks for the student (e.g., "on page 3, first paragraph, change to "). This process requires more of the grader's time and also sacrifices the clarity that is inherent with direct markup. Yet another approach that works when grading word processor documents is for the grader to make desired changes to the document using built-in tracking features (e.g., Microsoft Word's "Track Changes" tool). This changes the student's submitted work, however, and makes comparison of the revision to the original difficult.

Pen-Based Alternative
We have developed an alternative that combines the best of both approaches: It allows graders to write free-hand comments in colored "electronic ink" on the student's submission while preserving the paperless, electronic nature of the submission. This allows archiving of graded copies and electronic return of graded assignments to students. The approach makes use of pen-based tablets and Adobe Acrobat's Portable Document Format (PDF). Adobe's Acrobat software provides a means of marking up PDF documents with an electronic pen, so the resulting document resembles the original with the grader's notations overlaid, much like a graded paper assignment. Acrobat also allows creation of "stick-on notes" containing ASCII text. PDF documents may use external or embedded scripts written in Acrobat JavaScript, which allow access to and manipulation of document elements. Perhaps most importantly, Acrobat allows a PDF document to be digitally signed and locked, so that a grader's notations cannot be changed by another party afterwards.

Hardware Interfaces
Acrobat's electronic pen feature may be invoked by clicking a toolbar button and using the mouse as a pen, but such usage is obviously clumsy and inaccurate. Fortunately, inexpensive pen-based tablets are now available (e.g., the Wacom Graphire 2 graphics tablet costs less than $100 and features a cordless pen). The use of a pen-based tablet requires some hand-to-eye training, but is otherwise simple to use. More advanced tablet models are available, but their features are more useful for artists than graders, and their costs are comparable to PCs.

More expensive but easier to use are Tablet PCs, whose screens may be written on directly with a special stylus. Several manufacturers, including HP and Toshiba, released Tablet PCs in November 2002 to coincide with the release of the new Microsoft Windows XP Tablet Edition operating system. The industry's attention to these devices promises a new era when pen-based input will become a standard mode of computing.

Project DUPLEX
At Project DUPLEX (Drexel University Programming Learning EXperience), we have been investigating various ways to use technological advances to enhance the quality and delivery of large computer programming classes, while reducing costs of course administration. Using the WebCT course management system, we have found that electronic submission of assignments and lab exercises has made some aspects of course delivery simpler while confounding others. A typical offering of our Computer Programming I class comprises 250 students in two lecture sections and 10 to12 lab sections, with two instructors and eight to ten teaching assistants, lab assistants, and graders. Before using WebCT, we found management of assignments and lab exercises submitted on paper to be complex, error-prone, and time-consuming. Student assignments were frequently lost; student claims of lost assignments were difficult to verify. Sorting and distributing assignments to their respective graders also took time.

While these concerns were greatly alleviated through the use of electronic submission, we found a whole new set of concerns emerged. The greatest of these was the amount of time it now took graders to grade assignments. Downloading a set of assignments was time-consuming enough, but once obtained, our graders were either spending double the time previously needed to grade a computer program, or were not providing a desired level of feedback.

To alleviate this situation, we created Labrador, a Perl-based client that interfaces with WebCT to download student submissions in a variety of formats and convert them to PDF format, where they may be graded electronically. Labrador is still in development, but will be generally available as an aid in accessing assignments from WebCT, converting the assignments to appropriate formats for pen-based grading or submission or plagiarism detection software and other electronic processing.

We have experimented with several approaches to grading student assignments, and find a combination of pen markup and stick-on notes works best. Handwriting recognition is not an issue at present. We have used the Wacom tablets and Sony Vaio, and are experimenting with a new Tablet PC.

Good Old Days
The late computing science pioneer, Edsger W. Dijkstra once described himself in this way: "… he writes, in fact, so much that he cannot afford the use of time-saving devices such as word processors. He owns, however, several fountain pens, three of which are Mont Blancs, for which he mixes his own ink." As in Dijkstra's sentiments, we hope to find the use of the pen-based grading to be no more time-consuming than in the "good old days" of paper-based grading, while taking advantage of the obvious advantages of electronically submitted assignments.