Electronic Grading: When the Tablet is Mightier than the Pen
- By Jeffrey L. Popyack, Nira Herrmann
Technological advances often seem to introduce as many obstacles as improvements.
For instance, people seem to spend more time reading and writing electronic
mail than they ever did with regular mail. Likewise, VCR use is sufficiently
formidable to have inspired folk humor about how to program one.
And so it is with grading assignments that have been submitted electronically.
Electronic submissions of student assignments certainly provide many advantages
for the faculty member and graders. For instance, they are easier to manage
and keep track of than their paper counterparts, particularly as the number
of submissions increases. Submissions can be time-stamped automatically and
archived, thus minimizing the potential for disputes over lateness and lost
assignments and/or grades. Furthermore, archives can help resolve issues involving
academic dishonesty and/or plagiarism. Finally, paperless transactions are especially
useful when the assignments must be distributed to multiple locations, such
as to teaching assistants, graders, and plagiarism detection software.
of Electronic Submission
Assignments submitted electronically are not without drawbacks, however. People
are more familiar with manipulating paper documents and editorial markup, in
particular, has been much simpler with a paper document. Consider, for instance,
a student composition submitted as a text document where the grader wishes to
cite a particular passage as requiring revision. On a paper document, the grader
simply circles the offending passage, crosses out a few words, draws arrows
or other editorial markings, and writes remarks directly on the paper. The grader's
remarks (perhaps in red ink) stand out as distinct from the original submitted
work, and context is understood from their location on the paper. The grader
can work in close or cramped quarters, compare papers side-by-side, and easily
flip back and forth between pages.
With a document in electronic form, the grader must decide how best to convey
remarks while leaving the original submission intact. Users have tried a variety
of approaches, but to date, they have all posed some drawback in practical use.
One approach is to print a copy of the original document, then mark it up and
return it. This, however, places the burden of printing on the grader (prohibitive
in time and expense for large classes), and d'es not preserve an archival record
of the grader's remarks.
Another approach is for the grader to type a separate document with remarks
for the student (e.g., "on page 3, first paragraph, change to
"). This process requires more of the grader's time and also sacrifices
the clarity that is inherent with direct markup. Yet another approach that works
when grading word processor documents is for the grader to make desired changes
to the document using built-in tracking features (e.g., Microsoft Word's "Track
Changes" tool). This changes the student's submitted work, however, and makes
comparison of the revision to the original difficult.
We have developed an alternative that combines the best of both approaches:
It allows graders to write free-hand comments in colored "electronic ink" on
the student's submission while preserving the paperless, electronic nature of
This allows archiving of graded copies and electronic return
of graded assignments to students. The approach makes use of pen-based tablets
and Adobe Acrobat's Portable Document Format (PDF). Adobe's Acrobat software
provides a means of marking up PDF documents with an electronic pen, so the
resulting document resembles the original with the grader's notations overlaid,
much like a graded paper assignment. Acrobat also allows creation of "stick-on
notes" containing ASCII text. PDF documents may use external or embedded scripts
elements. Perhaps most importantly, Acrobat allows a PDF document to be digitally
signed and locked, so that a grader's notations cannot be changed by another
Acrobat's electronic pen feature may be invoked by clicking a toolbar button
and using the mouse as a pen, but such usage is obviously clumsy and inaccurate.
Fortunately, inexpensive pen-based tablets are now available (e.g., the Wacom
Graphire 2 graphics tablet costs less than $100 and features a cordless pen).
The use of a pen-based tablet requires some hand-to-eye training, but is otherwise
simple to use. More advanced tablet models are available, but their features
are more useful for artists than graders, and their costs are comparable to
More expensive but easier to use are Tablet PCs, whose screens may be written
on directly with a special stylus. Several manufacturers, including HP and Toshiba,
released Tablet PCs in November 2002 to coincide with the release of the new
Microsoft Windows XP Tablet Edition operating system. The industry's attention
to these devices promises a new era when pen-based input will become a standard
mode of computing.
At Project DUPLEX (Drexel University Programming Learning EXperience), we have
been investigating various ways to use technological advances to enhance the
quality and delivery of large computer programming classes, while reducing costs
of course administration. Using the WebCT course management system, we have
found that electronic submission of assignments and lab exercises has made some
aspects of course delivery simpler while confounding others. A typical offering
of our Computer Programming I class comprises 250 students in two lecture sections
and 10 to12 lab sections, with two instructors and eight to ten teaching assistants,
lab assistants, and graders. Before using WebCT, we found management of assignments
and lab exercises submitted on paper to be complex, error-prone, and time-consuming.
Student assignments were frequently lost; student claims of lost assignments
were difficult to verify. Sorting and distributing assignments to their respective
graders also took time.
While these concerns were greatly alleviated through the use of electronic
submission, we found a whole new set of concerns emerged. The greatest of these
was the amount of time it now took graders to grade assignments. Downloading
a set of assignments was time-consuming enough, but once obtained, our graders
were either spending double the time previously needed to grade a computer program,
or were not providing a desired level of feedback.
To alleviate this situation, we created Labrador, a Perl-based client that
interfaces with WebCT to download student submissions in a variety of formats
and convert them to PDF format, where they may be graded electronically. Labrador
is still in development, but will be generally available as an aid in accessing
assignments from WebCT, converting the assignments to appropriate formats for
pen-based grading or submission or plagiarism detection software and other electronic
We have experimented with several approaches to grading student assignments,
and find a combination of pen markup and stick-on notes works best. Handwriting
recognition is not an issue at present. We have used the Wacom tablets and Sony
Vaio, and are experimenting with a new Tablet PC.
Good Old Days
The late computing science pioneer, Edsger W. Dijkstra once described himself
in this way: "
he writes, in fact, so much that he cannot afford the use of
time-saving devices such as word processors. He owns, however, several fountain
pens, three of which are Mont Blancs, for which he mixes his own ink." As in
Dijkstra's sentiments, we hope to find the use of the pen-based grading to be
no more time-consuming than in the "good old days" of paper-based grading, while
taking advantage of the obvious advantages of electronically submitted assignments.
Herrmann N., Popyack J., Char B., Zoski P., Cera C., Lass R., Nanjappa
A. Redesigning Introductory Computer Programming Using Multi-level Online
Modules for a Mixed Audience. Submitted for publication (http://duplex.mcs.drexel.edu).
Cera C., Lass R., Char B., Popyack J., Herrmann N., Zoski P. Labrador:
A Tool for Automated Grading Support in Multi-Section Courses. WebCT 2002,
4th Annual Users Conference, Boston, Massachusetts, July 24-26, 2002.
Dijkstra, E., "On the Cruelty of Really Teaching Computing Science",
from "A Debate on Teaching Computing Science", Denning, P.,
Communications of the ACM Vol. 32, Issue 12 (Dec. 1989), pp 1397-1414.