Marist College

• By Meg Lloyd
• 07/23/13

In 2001, only 36 percent of US students who began bachelor's degree programs were able to complete them within four years, according to the US Department of Education. A decade later, the statistics are still cause for concern: According to the College Board Advocacy & Policy Center, in 2010 the US ranked 12th in the world for percentage of individuals aged 25 to 34 possessing an associate's degree or higher.

But higher ed is not taking those numbers lying down. The past 10 years have seen a big spike in initiatives surrounding learning analytics, with particular interest in the use of predictive models, "early academic alert" systems, and other programs designed to promote student success. From very early developments like Purdue University (IN)'s Course Signals project for early alerts and interventions (an Innovators awardee in 2011) to more recent, multi-institutional efforts like the PAR predictive analytics reporting framework project led by WCET, academic analytics has strong and growing research and community involvement.

The Marist OAAI team

While national efforts by institutions and associations have made progress in addressing the challenges of student success and degree completion, Marist College (NY) saw a need for an open ecosystem of analytics tools and resources. In 2011, Marist launched the Open Academic Analytics Initiative, supported initially by an Educause Next Generation Learning Challenges grant (funded in part by the Bill & Melinda Gates Foundation) and now maintained and furthered by the college. OAAI's main goal is to research the "portability" of predictive models, to create tools and resources that are applicable in diverse educational environments.

The OAAI system mines three historical data sets: student aptitude data, learning management system event-log data, and electronic gradebook data. Through a predictive model scoring process, the early alert system identifies at-risk students and allows instructors to deploy interventions to help those students succeed. Interventions can include awareness, tutoring, or online educational resources within the Open Academic Support Environment

OAAI's own development work is created using a fully open strategy: leveraging open software and resources, and creating a completely open learning analytics solution. OAAI's predictive model is available to other institutions under an open license, and all the system documentation and data integration flows have been released under open license. The initiative has a Sakai wiki page dedicated to the project where people can access all of the open materials, including the OAAI predictive model.

Corporate partners were selected for openness as well: Pentaho was chosen for its open source business intelligence tools; Sakai, the community source collaboration and learning environment, was selected as an open platform for OAAI, including rSmart technical support for Sakai. (rSmart's Sakai business is now owned by Asahi Net International.) IBM's SPSS was used to confirm analysis and to conduct technical testing.

Josh Baron, Marist's senior academic technology officer and OAAI project lead, reports, "Initial research findings have been very encouraging, showing statistically significant improvements in course grades and content mastery levels among those students in courses using the system."

One of the unexpected results Marist encountered relates to its study of the portability of the predictive model. "Given how dissimilar Marist is from our partner institutions, we were surprised to find that the model remained very accurate," says Baron, "which suggests that models could be used across a broad range of institution types--though we are working to customize models for specific types of institutions in hopes of even further increasing the accuracy."