Tales From the Front Lines of Adaptive Learning
Implementing adaptive learning technology in college courses can be an uphill struggle but well worth it, according to pioneering faculty members.
Adaptive learning platforms in higher education are starting to produce some promising results, yet the market is still in its infancy. Instructors who volunteer to be guinea pigs for pilot implementations often end up encouraged by the results — yet exhausted by how much work is involved in retrofitting their courses to the adaptive platforms.
Campus Technology interviewed several administrators and faculty members who have worked on adaptive projects about their experience. Here is what they told us:
Learning on the Go
When Shoreline Community College (WA) and Northeastern University (MA) received grant funding to work with vendor CogBooks on adaptive learning courses, Dutch Henry, a Shoreline English professor, volunteered to participate. "I have taught online for a long time but had not tried adaptive learning," he said. "I was curious about how it might help increase student success, because one of the issues we have at the two-year college level is that our success rate in online courses is very low."
Henry soon found that the project was an enormous amount of work for a variety of reasons. "The primary one was that until we worked with CogBooks, they had not done anything with colleges or universities at the scale of a quarter-long course," he said. The company's previous focus had been module-based workplace training, and it was difficult to adapt the platform for the large amounts of material covered in a full course, Henry explained.
Of the four Shoreline faculty who initially signed up, only two made it through that first round of course development. "One faculty member just threw her hands up and said 'I can't do this,'" Henry said with a laugh. "Another tried and is continuing to try but has yet to successfully put a portion of a class into CogBooks." It was very challenging — and not for lack of effort on anyone's part. CogBooks was helpful and persistent and they wanted to make it work, he added, but they just hadn't done it before. "And neither had we, so we were all learning on the go."
Henry noted that tailoring a course to adaptive learning is easier for subjects such as math, science and economics than it is for English and writing. "English is a difficult discipline to adapt to the model," he said. Students would submit a writing assignment, and then the system would show them models and ask them to self-assess whether their writing fit with the model and whether they understood the concept. If students answered no, the system would lead them back to more explanations of the concept. "So I had to build two or three layers of explanations, as you would in class if you gave an explanation and someone said, 'I don't get it.' You go to plan B. You have to build all that into the system," he said.
Henry is going on his third year of CogBooks use. "In economics and English, we have seen gains in terms of student persistence and success, if you define success by earning passing grades in courses," he said. "We are still studying how dramatic those improvements are and determining if we have enough students to say we have moved the needle. I hope that once we have enough students, we can say more definitively that CogBooks is the factor making a difference."
Thinking at the Curricular Level
Like at Shoreline, the shift to adaptive learning at the University of Central Florida turned out to be more work than expected — but worth it, according to Tom Cavanagh, associate vice president of distributive learning.
"We went in kind of naively thinking this was going to be analogous to how we do online learning," Cavanagh said. "We provide comprehensive training to faculty, provide media support and then the faculty drive the process." And while that is the case with the adaptive-learning courses, too, a different level of thinking is required to learn the new software system and figure out how to build adaptive content, he said. "The instructor has to think at the curricular level, not just the course level — where are the prerequisites and dependencies? What reinforces something later in the curriculum? It requires thought and content development that is not trivial."
UCF has been working with a product called RealizeIt from an Irish company called CCKF.
"We've built a team that will build courses on behalf of faculty under their guidance and with their approval, so they don't have to become experts on the software development side," Cavanagh explained. The two faculty members who have piloted the technology love it and want to keep teaching in the system, he said. Both of the instructors said they are not spending more or less time during the course — but how they spend their time is different. For example, the system is good at exposing students who might be struggling, Cavanagh noted. When a teacher logs in, the system might say, "Here are five students who are struggling and here are five who are excelling, who might need some extra engagement." That allows the instructor to focus on students who need the most attention right away. "They are able to use their time in more effective ways than they have in the past," Cavanagh said.
The limited data from a handful of courses with fewer than 200 students is encouraging, he added. "We found the performance at the module level for psychology was highly correlated to successful performance on a standard post-test that all students in all classes take. That was reassuring to see."
Another potential benefit of adaptive learning is that the system is designed to learn about individual students' strengths and weaknesses, pointed out Cavanagh. As a student, you take a pretest at the module level. Based on your performance, you might see a different path or have different nodes open to you in the module because you demonstrated mastery in the pretest. As you go through the course, the system checks itself and confirms or changes its confidence score about whether you know the material well. "As you go through the material, if you perform better on tests after seeing a video as opposed to reading passages, the system starts to learn video is better for you and by default if a video is available it feeds that to you first. The bigger the data set the more effective it becomes."
UCF is in the process of trying to spread the system's use to more courses. "First, we had to prove the concept. Now it is a matter of explaining it to faculty. A lot of them get it," Cavanagh said. "It is spreading, but not scaling as quickly as online learning has for us. We do think adaptive learning is a real thing. We have decided to invest in a centralized staff of instructional designers and course builders who can take the load off faculty members, so they can focus on being subject-matter experts and teaching the class."
Targeting Dual-Eligible Students
Dan McCoy, who is now chief learning officer for Adapt Courseware, spent time over the past few years working on adaptive learning at the University of Florida, in his previous role as senior director of e-learning, technology and creative services for the institution's College of Education.
The goal was to create online general education courses targeting high-achieving high school students interested in accelerated options such as dual enrollment, where they earn college credit. "We realized that to reach these kids, we needed to provide an online learning experience of exceptional quality," McCoy said. "Another challenge is if students are in dual enrollment, they are getting actual college credit and grades, and the experience becomes high-stakes for them."
The University of Florida worked with Adapt Courseware on a physics course that uses adaptive multimedia, including videos and animations, exercises, readings, assessments and analytics. "It involves short chunks of content like in a MOOC and frequent formative assessments," McCoy said, "so you capture the level of understanding of the material, as opposed to what we traditionally did, which was deliver six to eight weeks of material and then give a midterm exam."
The developers took all the lectures in the course and rewrote them for delivery on camera in 15-minute chunks and with a high-quality media experience.
Rather than fully automated remediation, the course is designed to provide a robust student services infrastructure, whereby students are closely monitored. If they are stuck in one of the lessons and can't progress, then the plan is to provide them with a live tutor. Meanwhile, advanced students could go through the material at their own pace. "We saw that as a scalable course model that was affordable," McCoy said. When McCoy left the university in early 2015, UF was in the midst of planning to roll out the adaptive courses for dual-enrolled students, he said.
Personalized Assignments
In order to give her students a more personalized learning experience, Rebecca Orr, a professor at two-year Collin College (TX), recently added Knewton Adaptive Follow-up (AFU) to the MasteringBiology curriculum from Pearson she has been using in her biology classes since 2009.
"I was frustrated because I have a wide spectrum of students, and with Mastering I had to create one-size-fits-all assignments," she said. "For some students it was perfect, but for others it was a long, laborious process." Orr was involved in helping to map the curriculum between Mastering and AFU. "That process raised a lot of questions about how successful the computer system would be in sensing students' content gaps versus what they were competent in," she said. "Could it take the place of a person in identifying misconceptions and telling students to go back?"
Once the curriculum mapping was done and the algorithms were ready in 2013, the question was, would it work? "I talked to my class and explained it was available," Orr remembered. "They were excited to try it."
In adopting AFU, Orr changed how she teaches the class by streamlining the assignments. "Rather than building to ultimate learning objectives, I took out all the basic stuff and shortened my assignments," she said. AFU has a feature that allows students to test out of follow-up if they make a 95 or better on the initial assignment. She gave them points in the course for completing AFU. If they test out, they get those points for free. "They thought that was phenomenal," she said. "But if they don't test out, AFU only sends you back to the areas where you show content gaps."
At first, student surveys revealed dissatisfaction with the system, but then Knewton started making adjustments as it got more student data, Orr said. "I heard fewer and fewer complaints, and more comments about it being really cool."
Orr compared the results of similar cohorts of students with and without AFU and saw a 5-point improvement in exam average by the end of the semester with AFU. "The key thing is that it allows students to have a more personalized learning experience with the content," she said, "and then come to me and I can work with them to think through the concepts rather than just parroting back basic knowledge."