Adaptive Learning Tech Linked to Better Test Performance

• By Dian Schaffhauser
• 01/22/19

Can adaptive technology help students encourage students to finish more assignments and do better on tests? That was the question researchers at the Center for Research and Reform in Education at Johns Hopkins University were tasked with answering in a sponsored research project. Knewton, which provides online and adaptive learning courses for undergraduate students in chemistry, economics, mathematics and statistics, contracted with the Center to run a study on the relationship between student use of Knewton's courses and subsequent success in the courses.

The news came out shortly before the company announced its own data analysis and reported that it was reducing its pricing.

According to the Johns Hopkins report, the study used Knewton data from the 2017 fall semester, which included:

• Average score on online tests and quizzes;
• The proportion of online assignments completed;
• The potential course dropout; and
• The numbers of adaptive items, learning objectives and assignments attempted.

"Overall," the researchers reported, "Knewton appears to be a useful tool for students. This study suggests a positive correlation among usage of Knewton, assignment completion and performance on online assessments."

Among the findings:

• The number of assignments students completed could predict student performance on online tests. A 10-percentage-point increase in the share of assignments completed (of those offered in the course) was linked to an increase in average student performance of 1.4 percentage points; and among those assignments attempted by students, a 10-percentage-point increase in the proportion of assignments completed was associated with an increase in average student performance of 1.2 percentage points.
• Completion of a single learning objective was tied to a 6.6-percentage-point increase in the average score for all of the quiz or test items related to that learning objective.
• Attempting an additional 250 adaptive items (beyond the average) was associated with an increase in the proportion of assignments completed by 14 percentage points.
• When participants finished an additional 10 percent (beyond the average) of the first quarter of assignments in the class, they were more likely to remain engaged in the work throughout the duration of the course by 4 percentage points.

These results were consistent overall across courses in different subjects and even for students of different ability levels. The one exception, the research noted, was that students of higher ability finished their assignments faster than students with lower ability, "which is expected given the adaptive nature of Knewton."

In its own analysis of data generated during the fall 2018 semester, Knewton reported the following:

• Students spent, on average, 2.8 hours per week in Alta;
• About 38 percent of the time, students would engage in the personalized instruction provided by Alta if they got a question wrong;
• Most students — 88 percent — who began an Alta assignment would also complete it; and
• Alta was associated with higher test scores; students who finished Alta assignments did 22 percent better on tests.

According to Knewton, the Alta line is used by instructors in 250-plus colleges and universities.

Knewton recently introduced a bundle deal for an all-access option. Altapass gives students two years of access to Alta courses within a single subject for $79.95. The company also reduced the cost of a single course Alta subscription from $44 to $39.95. The cost of a monthly subscription to a lone Alta course remains $9.95 per month. The new pricing goes into effect in time for the fall 2019 semester.

The Johns Hopkins report is openly available on the Knewton website. Coverage of data analysis from Knewton is available on a company blog article.