Automating Part of the Grading Process
Virginia Commonwealth University is using an automated grading tool to give students the coding practice they need without overwhelming instructors.
Learning to code well requires many hours of practice. However, the opportunities students have to apply what they're learning are limited by how many projects the instructor can reasonably assign and grade. It can be time-consuming for instructors to review each student's work and give constructive feedback, which restricts the amount of coding practice that students receive in their courses.
Debra Duke, director of the undergraduate computer science department at Virginia Commonwealth University, found a solution: She uses a platform called Gradescope to save time and automate parts of the review and feedback process. This allows Duke to assign more projects and give more meaningful, individualized feedback, and her students now get 50 percent more coding practice than they did before.
As an added bonus, Duke finds that her students are able to solve problems and take more responsibility for debugging their programs — and when they can't figure out something on their own, their questions are more specific.
We asked Duke how automated grading has led to more frequent and meaningful coding practice for her students.
Campus Technology: What courses do you teach?
Debra Duke: My primary area of focus is teaching a Computer Science 2 course, which is the second course that computer science majors take after their Intro to Programming course. It's programming intensive, with the goal that students leave at the end of the semester as programmers.
I also teach an Intro to Software Engineering course, which students usually take in their sophomore year. Students learn Agile programming concepts and they work in teams to create an Android app as their semester-long project. In addition, I teach an upper-level course in database theory.
CT: How many coding projects do you typically assign?
Duke: My CS 2 students now complete six main projects per semester. I used to be able to assign only four before I started using Gradescope. It's a big plus that I can actually assign more projects now.
They also do smaller projects in their weekly labs. The labs are two hours long, and during the labs they complete smaller projects that have more scaffolding, more code that is already written. They have to complete the code to get the program to work.
CT: What was your process for reviewing students' work before, and why was it so labor-intensive?
Duke: My CS 2 class has 80 to 90 students. It took a lot of time for me to download each student's program from our learning management system, save it on my computer with a new file name, and run it through the test suite to check their work. I had scripts for doing this, but it was still largely a manual process.
Even with the limited automation that I had, I would have to create my own spreadsheet with grades and upload the spreadsheet into the LMS. It was very time-consuming, which was why I could only assign four projects per semester.
CT: What does this process look like now? How does it compare?
Duke: The beauty of Gradescope is that it allows programming instructors to run students' coding projects through test suites at scale. Instructors can upload their own test suites, and Gradescope also includes built-in autograders for Java and Python. Because my CS 2 students program in Java, I use the built-in autograder to evaluate students' work.
When students submit a programming project to the Gradescope platform, the autograder executes a collection of tests on the students' code and provides a report to them on which tests pass or fail. It also gives them a score report. When I set up the assignment, I define the number of points that can be earned for each test within the autograder; for example, I can say that 80 out of 100 points for a certain project are earned via the tests run by the autograder.
Students earn the remaining points based on a visual expectation of the code for things like coding style, readability and so on. I can quickly scroll through each student's source code within Gradescope to grade these aspects of the project myself. It's really much easier than downloading multiple files from my school's LMS or opening 80 GitHub repositories.
CT: Can you describe the impact that using Gradescope is having in your classes?
Duke: Once Gradescope has autograded students' work and I have added my own review and feedback, I can just hit one button and have students' grades integrated with the LMS. That saves me a lot of time.
Gradescope also gives students a chance to submit their work early and get feedback on their project. They can see how many points they're earning through the autograder, and it guides them on revising their project and resubmitting. Students are actually doing more programming as a result, because it's not a one-and-done process where they just submit their work and wait for my grade. They keep working on their code to improve it (and their grade). So, in addition to completing 50 percent more projects, they're getting a lot more programming experience and feedback. That's a good thing.
I've also added autograding to my students' labs. When students finish a lab assignment, they submit it to Gradescope and get immediate feedback. Once they're satisfied that they've gotten the points they want, they leave. I can go back and upload their scores automatically from Gradescope to the LMS. This frees up my teaching assistants to help students debug their code, rather than just checking off whether students have done the assignment. I'd rather have my TAs' limited time spent teaching instead of reviewing students' work.
Also, I find that my students now ask more meaningful questions. Because they get feedback from the autograder as to which methods are failing the test, the questions they ask me about their projects are more directed. It's not just, "I don't know why this is failing." Instead, they'll say something like, "In my Contains All method, I'm missing on these three tests. Can you review this method and help me see where it's going wrong?"
They're able to direct their own debugging, and they're getting more autonomy to try to solve their own problems. Plus, they're getting more coding practice as well, because I'm able to assign more projects in my classes. It's a win-win.