Community Colleges | Feature

Community Colleges Leverage Predictive Analytics

A Q&A with two community college leaders in learning analytics

With today's emphasis on student success and retention, along with more restrictive funding that’s increasingly performance-based, community colleges are sharpening their strategic decision making and gearing up their toolboxes with predictive analytics.

At Rio Salado College, Interim Director of Research, Planning and Development Daniel Huston follows the progress of more than 41,000 online students via Rio PACE, a learning analytics system that's in place in nearly all courses offered at the college, while he makes extensive plans for even more sophisticated student performance tracking and intervention systems. David Shulman, AVP for online and instructional technology at Broward College is getting his institution started on predictive analytics, and ramping up fast by mapping all gen ed courses to learning outcomes--providing data for an analytics program linked to Broward's Desire2Learn LMS. Both institutions have joined as participants in the Predictive Analytics Reporting (PAR) Framework Project, a multi-institutional effort led by WCET that is creating a shared data resource containing more than a million de-identified student records for the purpose of creating new models that institutions can leverage for their predictive analytics initiatives. Ultimately, more data means more student success. Campus Technology talked to the two leaders to find out why.

Mary Grush: Dan, how does Rio Salado College use predictive analytics, now?

Daniel Huston: At Rio we have a system we call Rio PACE (Progress and Course Engagement), that's built into our LMS to identify students who are at risk of not successfully completing a particular course. So when a student logs in to the course through the LMS, they have an indicator--green, yellow, or red--of how well they are doing compared to students who have completed the course within the last year.

The instructor has a similar indicator in his or her roster. If you are teaching an in-person course, there are lots of subtle cues you can pick up on as to how well the students are grasping materials. But in an online environment sometimes you don't get those cues, so Rio PACE identifies the major factors that can show whether or not a student will be successful, compared to other students who have completed the course.

We have operationalized this as a tool to help both students and instructors gauge how well they are doing. [see "Monitoring the PACE of Student Learning: Analytics at Rio Salado College"]

Grush: David, what is Broward College's current involvement in analytics?

David Shulman: Broward is really at the beginning stages. Along with our recent migration to the Desire2Learn LMS (we are in our second term using it), we also purchased the analytics program. So at this point we have a task force that is working on what kind of reports we want out of the analytics program. This is all very timely for us though--we have a new strategic plan, and the goals include improving the first year experience as well as retention and persistence for students. For us as an online program, we want to see not just how students have done, but also to be able to predictively see where they are having, or headed for, problems.

Grush: Dan, what are some of your plans at Rio to extend your use of predictive analytics?

Huston: There are three more areas, beyond the student risk indicators, where we'd like to develop predictive analytics: First, building on our current risk indicators to create a model for students that's above the course level; then creating an academic advising component; and finally closing the loop on capturing interventions.

It's going to be particularly important to build a risk model that works above the course level. In predictive modeling, we want to look at lots of data because there are subtleties there that you might not notice if you are looking at just a handful of data points. So, for example, with a given student, it may be easier to identify whether they are at risk by looking across their courses: There may be some risk behaviors shown in individual classes that are not at a level you'd categorize as at risk, but when you aggregate behavior across a student's classes, you can see clearly that the student is in fact at risk.

Also, right now we have an indicator that tells both the student and the instructor whether a student is potentially at risk. Up to this point, we've provided the indicator to the student and instructor, but we haven't captured any information about the intervention or follow up. Hopefully capturing this information will help us to identify best practices at the student level, the instructor level, the course level, or at the subject level. We're very excited about that.

Grush: Dan, Rio is one of the original six institutions participating in the PAR Project. What are some of the benefits your institution has seen from participating in PAR, and what are your expectations for continuing as a partner?

Huston: The main thing in going forward with PAR is that having data from so many participating institutions will allow us to identify patterns that aren't detectable with data from our Rio students only. This will help us make our own models at Rio more accurate.

And besides the scope of the data available, partnering with other institutions through PAR is a great opportunity to go back and look at our own data and the processes surrounding it, and make sure that our data is as relevant as it can be. For example, when the original six participating institutions were first developing a list of variables for PAR, we found that Rio had no response for a couple of the variables. We weren't collecting or capturing that particular data in a way that made sense for the project. It was a real eye-opening experience to consider our data from a new perspective.

It's exciting for me, too, just to be able to talk to other partners. I'm the only one currently working on predictive analytics at Rio. We are looking to expand the team, but for now it's good to have so many colleagues in predictive analytics in the PAR Project.

Grush: David, I know that Broward is a new member of the PAR Project. What influenced Broward to join PAR?

Shulman: For us, PAR is coming at a perfect time, because it's going to help us identify what we ourselves want from Broward's own analytics program. Working with the other institutions in PAR is going to be very useful in working out what it is we need to discover in the kind of analytics modeling we are going to be doing ourselves.

PAR also comes at a time when all our institutions are looking at what we need to know, because in one form or another we are all getting to a point where our financing is going to become performance-based, at least as far as state funding goes for the public institutions. So it's a good time to start analyzing and making sure that students are successful, and to be able to step in where students are struggling.

It's very interesting to be able to listen to all the other PAR institutions. Some institutions are a little bit further with analytics, and some have slightly different ways they will use their analytics, but we're all pretty much headed to the same thing: a focus on student success and completion.

Grush: David, are there more specific things you've gained already or benefits you are expecting from Broward's participation in PAR?

Shulman: At Broward, for our learning analytics initiative, we're starting off with taking all of our general education courses and doing learning outcomes mapping with the analytics tool. Soon we will be able to look and see how students are doing at any given moment. I think that what PAR is going to add to this is that predictive look at the types of students we have, which will give us the ability to get in before problems occur and help the students be successful. This is going to be a big boost to the power of our analytics at Broward.

At our kickoff meeting for PAR participants about a week ago, Sinclair Community College was talking about one particular program they have that is very similar, and their figures were very encouraging. It was good to hear how helpful it had been to students, especially at the community college level, where students may need a little bit more intervention, I would say.

And more technically, by looking at the PAR variables, we as an institution are starting to see where we are weak and missing certain datasets. It's been a very interesting experience. It was particularly interesting for our head of institutional research, who looked at the initial datasets and realized the questions we were not yet asking ourselves.

Grush: And all this is being driven by a sharing of data at scale…

Shulman: I don’t think there has ever been anything that's been done with this volume of records and this number of diverse profiles of institutions--this is nothing like just one institution doing research and coming up with findings; it is a lot stronger. It's unprecedented in terms of scope and scale.

[Editor’s note: For more on the expanded PAR Project, see CT's interview with WCET Executive Director Ellen Wagner.]

comments powered by Disqus