Community Colleges | Feature
Community Colleges Leverage Predictive Analytics
A Q&A with two community college leaders in learning analytics
With today's emphasis on student success and retention,
along with more restrictive funding that’s increasingly performance-based,
community colleges are sharpening their strategic decision making and gearing
up their toolboxes with predictive analytics.
At Rio Salado College, Interim Director of Research, Planning and Development
Daniel Huston follows the progress of more than 41,000 online students via Rio
PACE, a learning analytics system that's in place in nearly all courses offered
at the college, while he makes extensive plans for even more sophisticated
student performance tracking and intervention systems. David Shulman, AVP for
online and instructional technology at Broward College is getting his
institution started on predictive analytics, and ramping up fast by mapping all
gen ed courses to learning outcomes--providing data for an analytics program
linked to Broward's Desire2Learn LMS. Both institutions have joined as
participants in the Predictive Analytics Reporting (PAR) Framework Project, a multi-institutional effort led by WCET that
is creating a shared data resource containing more than a million de-identified
student records for the purpose of creating new models that institutions can
leverage for their predictive analytics initiatives. Ultimately, more data
means more student success. Campus Technology talked to the two leaders to find out why.
Mary Grush: Dan, how does Rio Salado College use predictive analytics, now?
Daniel Huston: At Rio we have a system we call Rio PACE (Progress and Course Engagement), that's built into our LMS to identify students who are at risk of not successfully
completing a particular course. So when a student logs in to the course through
the LMS, they have an indicator--green, yellow, or red--of how well they are
doing compared to students who have completed the course within the last year.
The instructor has a similar indicator in his or her roster.
If you are teaching an in-person course, there are lots of subtle cues you can
pick up on as to how well the students are grasping materials. But in an online
environment sometimes you don't get those cues, so Rio PACE identifies the
major factors that can show whether or not a student will be successful,
compared to other students who have completed the course.
We have operationalized this as a tool to help both students
and instructors gauge how well they are doing. [see "Monitoring the PACE of Student Learning: Analytics at Rio Salado College"]
Grush: David, what is Broward College's current involvement in analytics?
David Shulman: Broward is really at the beginning stages. Along with our recent migration to the Desire2Learn LMS (we are in our second term using it), we also purchased
the analytics program. So at this point we have a task force that is working on
what kind of reports we want out of the analytics program. This is all very
timely for us though--we have a new strategic plan, and the goals include improving
the first year experience as well as retention and persistence for students.
For us as an online program, we want to see not just how students have done,
but also to be able to predictively see where they are having, or headed for,
problems.
Grush: Dan, what are some of your plans at Rio to extend your use of predictive analytics?
Huston: There are three more areas, beyond the student risk indicators, where we'd like to develop predictive analytics: First, building on our current risk indicators to
create a model for students that's above the course level; then creating an
academic advising component; and finally closing the loop on capturing
interventions.
It's going to be particularly important to build a risk
model that works above the course level.
In predictive modeling, we want to look at lots of data because there are
subtleties there that you might not notice if you are looking at just a handful
of data points. So, for example, with a given student, it may be easier to
identify whether they are at risk by looking across their courses: There may be some risk behaviors shown in individual classes that are not at a level you'd categorize as at
risk, but when you aggregate behavior across a student's classes, you can see
clearly that the student is in fact at risk.
Also, right now we have an indicator that tells both the
student and the instructor whether a student is potentially at risk. Up to this
point, we've provided the indicator to the student and instructor, but we
haven't captured any information about the intervention or follow up. Hopefully
capturing this information will help us to identify best practices at the
student level, the instructor level, the course level, or at the subject level.
We're very excited about that.
Grush: Dan, Rio is one of the original six institutions participating in the PAR Project. What are some of the benefits your institution has seen from participating in PAR, and what are your expectations for continuing as a partner?
Huston: The main thing in going forward with PAR is that having data from so many participating institutions will allow us to identify patterns that aren't detectable with
data from our Rio students only. This will help us make our own models at Rio
more accurate.
And besides the scope of the data available, partnering with
other institutions through PAR is a great opportunity to go back and look at
our own data and the processes surrounding it, and make sure that our data is
as relevant as it can be. For example, when the original six participating
institutions were first developing a list of variables for PAR, we found that
Rio had no response for a couple of the variables. We weren't collecting or
capturing that particular data in a way that made sense for the project. It was
a real eye-opening experience to consider our data from a new perspective.
It's exciting for me, too, just to be able to talk to other
partners. I'm the only one currently working on predictive analytics at Rio. We
are looking to expand the team, but for now it's good to have so many
colleagues in predictive analytics in the PAR Project.
Grush: David, I know that Broward is a new member of the PAR Project. What influenced Broward to join PAR?
Shulman: For us, PAR is coming at a perfect time, because it's going to help us identify what we ourselves want from Broward's own analytics program. Working with the other
institutions in PAR is going to be very useful in working out what it is we
need to discover in the kind of analytics modeling we are going to be doing
ourselves.
PAR also comes at a time when all our institutions are
looking at what we need to know, because in one form or another we are all
getting to a point where our financing is going to become performance-based, at
least as far as state funding goes for the public institutions. So it's a good
time to start analyzing and making sure that students are successful, and to be
able to step in where students are struggling.
It's very interesting to be able to listen to all the other
PAR institutions. Some institutions are a little bit further with analytics,
and some have slightly different ways they will use their analytics, but we're
all pretty much headed to the same thing: a focus on student success and
completion.
Grush: David, are there more specific things you've gained already or benefits you are expecting from Broward's participation in PAR?
Shulman: At Broward, for our learning analytics initiative, we're starting off with taking all of our general education courses and doing learning outcomes mapping with the analytics tool. Soon we will be able to look and see how students are doing at
any given moment. I think that what PAR is going to add to this is that
predictive look at the types of students we have, which will give us the
ability to get in before problems occur and help the students be successful.
This is going to be a big boost to the power of our analytics at Broward.
At our kickoff meeting for PAR participants about a week
ago, Sinclair Community College was talking about one particular program they
have that is very similar, and their figures were very encouraging. It was good
to hear how helpful it had been to students, especially at the community
college level, where students may need a little bit more intervention, I would
say.
And more technically, by looking at the PAR variables, we as
an institution are starting to see where we are weak and missing certain
datasets. It's been a very interesting experience. It was particularly
interesting for our head of institutional research, who looked at the initial
datasets and realized the questions we were not yet asking ourselves.
Grush: And all this is being driven by a sharing of data at scale…
Shulman: I don’t think there has ever been anything that's been done with this volume of records and this number of diverse profiles of institutions--this is nothing like just one institution doing research and coming up with findings; it is a lot
stronger. It's unprecedented in terms of scope and scale.
[Editor’s note: For more on the expanded PAR Project, see
CT's interview with WCET Executive Director Ellen Wagner.]
