Opening Up Learning Analytics for the Community College
A Q&A with Josh Baron and JoAnna Schilling
The Open Academic Analytics Initiative, a project supported
by an EDUCAUSE Next Generation Learning Challenge grant funded primarily
through the Bill and Melinda Gates Foundation, has developed a predictive model
for learning analytics using open source technologies. Marist College is the
lead institution on the grant, partnering with Cerritos College and College of
the Redwoods (both community colleges), along with Savannah State University
and North Carolina A&T State University (both HBCUs), and one corporate
partner, rSmart.
The OAAI project seeks to engender an ecosystem for learning
analytics based on open source technologies. OAAI conducts research into
scaling factors for learning analytics, the portability of predictive models
among different types of academic institutions, and intervention strategies
that leverage Open Educational Resources. The institutional partners on the
grant have quickly ramped up and piloted the OAAI predictive model since its
development after the first wave of funding in 2011, and they are beginning to
share research findings and data as well as anecdotal information and the
experiences of institutional partners. CT spoke with PI Josh Baron, Marist's
Senior Academic Technology Officer, and JoAnna Schilling, Vice President of
Academic Affairs at OAAI partner Cerritos College.
Mary Grush: Josh,
what are the OAAI's main objectives?
Josh Baron: There
are two major objectives. One is to develop an open ecosystem for learning
analytics; to build a learning analytics system that leverages open source
technologies. The other objective is in the area of research related to scaling
learning analytics on a national level--the question of portability of
predictive models to different academic contexts is especially pertinent to
that research.
Grush: Very
generally, what are you doing to research portability?
Baron: We want to
see learning analytics go from what has been fairly isolated--although very
successful--pockets of innovation (for example, Purdue's Course Signals), to scaled
learning analytics at hundreds or thousands of different institutions across
the country. We are trying to understand how predictive models built at one
institution might be deployed successfully in a very different academic context
at another institution. As part of our research, we developed a predictive model
at Marist College (based on data related to our students and our courses), and then we deployed that model at our two
partner community colleges, Cerritos and College of the Redwoods. Understanding
how well that model performs when we move it from one academic context to
another helps us discover ways of scaling learning analytics.
We have analyzed the data in our research to date, and we
have shown that the model we built using Marist data remained in the 60-75
percent accuracy range when we deployed it at these other types of
institutions. This research result has shown us that models are more portable
than we first thought.
Grush: Do you think
open predictive models--that use open source tools--can also help scale
learning analytics by removing some of the cost barriers?
Baron: Yes. We
developed our OAAI predictive model (and the technology needed to deploy that
model) with an open source set of tools, so that anyone could take the model
that we developed and use it without incurring a large licensing fee.
For example, we are using Sakai as our open source
collaboration and learning environment--as our LMS--and we use the Pentaho open
source suite of tools for data mining, reporting, and data integration.
Further, we are going to offer the OAAI predictive model under a Creative
Commons open source license. [For further information about this model, see the
Sakai wiki page.]
Grush: What are some
of the other open source elements of this strategy?
Baron: Another major
one is incorporating Open Educational Resources in our
intervention strategies, and we have found that to have a
positive, statistically significant impact on student course grades and
content mastery.
Grush: JoAnna, isn't
that how Cerritos College originally got involved in OAAI--through its interest
in OERs?
JoAnna Schilling:
Yes. Our involvement in OAAI actually grew out of another Gates grant that
Cerritos had applied for and received, focusing on OERs. Cerritos is very
invested in OERs. It's important to the administration at Cerritos to continue
to find ways to lower costs for students, and while lowering cost was the
original impulse, we also found some exciting data that shows that when we
deliver learning material in a different and non-traditional way, the students
get excited about it--and that is reflected in our retention rates and in the
students' engagement.
The OAAI grant also had Gates funding, and many of the
partner schools that Cerritos had in their earlier grant also became part of
the OAAI grant. Being included in this additional program, as a part of the
OAAI, has been very exciting.
Grush: So far, how
has your institution used the OAAI data?
Schilling: This is
only the second semester that we've participated in OAAI. So we still need to
analyze the data more to help determine how we will be acting on it. But our
main interest is in increasing our completion rate. Anecdotally, what we have
already seen is that students who have been at high risk are moving closer to
moderate risk; and moderate risk students are moving closer to low risk.
Grush: And that's
based on interventions that have occurred as a result of the OAAI predictive
model?
Schilling: We can't
be 100 percent sure of all of the things that might factor into this
improvement, but the increased interaction that faculty are providing students
[in their interventions] certainly seems to be what the students appreciate
most. So, just providing the resources isn't enough; it's the instructor
interaction with the students, and the instructors' guidance to the resources
that the students need that can make a difference.
Grush: JoAnna, do
you see learning analytics as a value for community colleges?
Schilling: I'm very
interested in how technology can help us engage our students. One of the
biggest challenges in community colleges is lack of student engagement. There
are many barriers in the way our system is set up, that make us much different
from a four-year institution. We are not a residence institution; our students
come from all different walks of life; our admissions is open enrollment. So,
really knowing who our students are and discovering techniques to engage them
is our biggest challenge.
I do see analytics helping us to identify things that
students may respond to. All of the research tells us that students are at
great risk if we don't engage them the first semester they come to our
institution. So what I am interested in, and what our faculty is interested in,
is how we can identify certain behaviors, both on the part of the institution
and on the part of the student, that can help us to interact and to engage our
students sooner.
Grush: Josh, is the
OAAI open source strategy aimed at helping institutions get on track with
learning analytics quickly?
Baron: Yes
absolutely. OAAI really is focused on a very rapid deployment that can scale.
We want to move colleges and universities forward with learning analytics that
will help address problems like completion rates nationally. So from day one we
were focused not just on theory, but on developing and deploying practical
learning analytics technologies on an operational level that institutions can
deploy now.
Of course, many of the other benefits of open source apply
to this project and will come from the release of the OAAI predictive model
under an open license. These would of course include the ability for others to
take the model and modify/enhance it. But the notion of leveraging open source
in a way that will, as you say, help get institutions on track with learning
analytics quickly is very compelling at a time when institutions need to
address problems with data.