Community Colleges | Feature

Opening Up Learning Analytics for the Community College

A Q&A with Josh Baron and JoAnna Schilling

The Open Academic Analytics Initiative, a project supported by an EDUCAUSE Next Generation Learning Challenge grant funded primarily through the Bill and Melinda Gates Foundation, has developed a predictive model for learning analytics using open source technologies. Marist College is the lead institution on the grant, partnering with Cerritos College and College of the Redwoods (both community colleges), along with Savannah State University and North Carolina A&T State University (both HBCUs), and one corporate partner, rSmart.

The OAAI project seeks to engender an ecosystem for learning analytics based on open source technologies. OAAI conducts research into scaling factors for learning analytics, the portability of predictive models among different types of academic institutions, and intervention strategies that leverage Open Educational Resources. The institutional partners on the grant have quickly ramped up and piloted the OAAI predictive model since its development after the first wave of funding in 2011, and they are beginning to share research findings and data as well as anecdotal information and the experiences of institutional partners. CT spoke with PI Josh Baron, Marist's Senior Academic Technology Officer, and JoAnna Schilling, Vice President of Academic Affairs at OAAI partner Cerritos College.

Mary Grush: Josh, what are the OAAI's main objectives?

Josh Baron: There are two major objectives. One is to develop an open ecosystem for learning analytics; to build a learning analytics system that leverages open source technologies. The other objective is in the area of research related to scaling learning analytics on a national level--the question of portability of predictive models to different academic contexts is especially pertinent to that research.

Grush: Very generally, what are you doing to research portability?

Baron: We want to see learning analytics go from what has been fairly isolated--although very successful--pockets of innovation (for example, Purdue's Course Signals), to scaled learning analytics at hundreds or thousands of different institutions across the country. We are trying to understand how predictive models built at one institution might be deployed successfully in a very different academic context at another institution. As part of our research, we developed a predictive model at Marist College (based on data related to our students and our courses), and then we deployed that model at our two partner community colleges, Cerritos and College of the Redwoods. Understanding how well that model performs when we move it from one academic context to another helps us discover ways of scaling learning analytics.

We have analyzed the data in our research to date, and we have shown that the model we built using Marist data remained in the 60-75 percent accuracy range when we deployed it at these other types of institutions. This research result has shown us that models are more portable than we first thought.

Grush: Do you think open predictive models--that use open source tools--can also help scale learning analytics by removing some of the cost barriers?

Baron: Yes. We developed our OAAI predictive model (and the technology needed to deploy that model) with an open source set of tools, so that anyone could take the model that we developed and use it without incurring a large licensing fee.

For example, we are using Sakai as our open source collaboration and learning environment--as our LMS--and we use the Pentaho open source suite of tools for data mining, reporting, and data integration. Further, we are going to offer the OAAI predictive model under a Creative Commons open source license. [For further information about this model, see the Sakai wiki page.]

Grush: What are some of the other open source elements of this strategy?

Baron: Another major one is incorporating Open Educational Resources in our intervention strategies, and we have found that to have a positive, statistically significant impact on student course grades and content mastery.

Grush: JoAnna, isn't that how Cerritos College originally got involved in OAAI--through its interest in OERs?

JoAnna Schilling: Yes. Our involvement in OAAI actually grew out of another Gates grant that Cerritos had applied for and received, focusing on OERs. Cerritos is very invested in OERs. It's important to the administration at Cerritos to continue to find ways to lower costs for students, and while lowering cost was the original impulse, we also found some exciting data that shows that when we deliver learning material in a different and non-traditional way, the students get excited about it--and that is reflected in our retention rates and in the students' engagement.

The OAAI grant also had Gates funding, and many of the partner schools that Cerritos had in their earlier grant also became part of the OAAI grant. Being included in this additional program, as a part of the OAAI, has been very exciting.

Grush: So far, how has your institution used the OAAI data?

Schilling: This is only the second semester that we've participated in OAAI. So we still need to analyze the data more to help determine how we will be acting on it. But our main interest is in increasing our completion rate. Anecdotally, what we have already seen is that students who have been at high risk are moving closer to moderate risk; and moderate risk students are moving closer to low risk.

Grush: And that's based on interventions that have occurred as a result of the OAAI predictive model?

Schilling: We can't be 100 percent sure of all of the things that might factor into this improvement, but the increased interaction that faculty are providing students [in their interventions] certainly seems to be what the students appreciate most. So, just providing the resources isn't enough; it's the instructor interaction with the students, and the instructors' guidance to the resources that the students need that can make a difference.

Grush: JoAnna, do you see learning analytics as a value for community colleges?

Schilling: I'm very interested in how technology can help us engage our students. One of the biggest challenges in community colleges is lack of student engagement. There are many barriers in the way our system is set up, that make us much different from a four-year institution. We are not a residence institution; our students come from all different walks of life; our admissions is open enrollment. So, really knowing who our students are and discovering techniques to engage them is our biggest challenge.

I do see analytics helping us to identify things that students may respond to. All of the research tells us that students are at great risk if we don't engage them the first semester they come to our institution. So what I am interested in, and what our faculty is interested in, is how we can identify certain behaviors, both on the part of the institution and on the part of the student, that can help us to interact and to engage our students sooner.

Grush: Josh, is the OAAI open source strategy aimed at helping institutions get on track with learning analytics quickly?

Baron: Yes absolutely. OAAI really is focused on a very rapid deployment that can scale. We want to move colleges and universities forward with learning analytics that will help address problems like completion rates nationally. So from day one we were focused not just on theory, but on developing and deploying practical learning analytics technologies on an operational level that institutions can deploy now.

Of course, many of the other benefits of open source apply to this project and will come from the release of the OAAI predictive model under an open license. These would of course include the ability for others to take the model and modify/enhance it. But the notion of leveraging open source in a way that will, as you say, help get institutions on track with learning analytics quickly is very compelling at a time when institutions need to address problems with data.

comments powered by Disqus