Analytics | Feature

12 Essentials of Prescriptive Analytics for Student Success

• By Dian Schaffhauser
• 06/25/14

While predictive analytics have been an ed tech buzzword in recent years, they are but a midpoint in the evolution of data analytics in higher education. What began as descriptive analytics, the analysis of historical data to understand what has happened in the past, has matured into predictive analytics, the use historical data to develop models for helping to predict the future. Now, prescriptive analytics takes the prediction and prescribes recommendations or actions to influence what ends up happening in the future. It works by developing business rules that kick into action when certain conditions are present. For example, a prescriptive analytics-driven learning management system could recommend additional material or Web sites to a student with poor performance pertaining to a specific topic.

Analytics is an area that Rajeev Bukralia is "passionate" about. He believes it has the potential "to help institutional leaders make smarter and faster decisions." Bukralia, who is the CIO and associate provost for information services at the University of Wisconsin-Green Bay, is also a researcher in the fields of analytics, data science and information systems. His research work focuses on how analytics can solve real-world problems in higher education and other sectors.

Here Bukralia shares 12 lessons for finding success with prescriptive analytics.

1) Carefully Choose the Problem to Solve

Prescriptive analytics is well suited for a variety of problems besetting higher education -- student retention, enrollment management, prospect analysis, improving learning outcomes and curricular planning among them. The primary considerations, said Bukralia, are that the problem is "well suited" for data analytics and that there's an "adequate amount of high-quality data to really analyze that problem." He advised against taking on a business problem that is so ambitious, the project itself will take too long to deliver, opening up the possibility that the problem could become irrelevant as time goes by.

2) The Model Needs Cross-Validation

A predictive model can be combined with business rules to create a prescriptive model with automated recommendations. This model not only needs to show a "high predictive accuracy," but it needs to be cross-validated "in terms of efficacy," said Bukralia. That may mean using different data sets or even taking the model to a different organization "to see if the predictive accuracy of the model is what we need to have." He emphasized that controls will probably need to be put in place to accommodate for "certain kinds of variables because each institution is different. You have to account for that uniqueness."

3) Business Needs to Drive the Work

Although the IT or IS organization can help facilitate the work of prescriptive analytics, ultimately the initiative must be led by people who are closest to the problem and the data. For instance, prescriptive analytics for enrollment should be led by the enrollment office. "They are the people who know the problem they are facing. They are the people who understand their data," he explained.

4) Involve Practitioners in Defining Business Rules

The business rules that make up the output of prescriptive analytics need to be formulated, Bukralia insisted, with input from various stakeholders in the organization. When building a model to improve student retention, for instance, input should be sought from faculty, advisers, institutional researchers and student retention staff. "In order to really develop a system that's insightful, you have to take advantage of the knowledge that people have who are working with students on a day-to-day basis," he said. "They can provide these insights based on their knowledge and practical experience."

5) Data Can't Be Used Indiscriminately

Schools are becoming accustomed to tapping into the data generated by student information systems and learning management systems to address challenges such as identifying at-risk students. Other sources of data may also prove useful, but they come with controversy. Bukralia cited less obvious forms of data that are generated by mobile device usage, campus cards, social media and sensor technologies.

"For example, if an on-campus student is not using the dining service as often, that [may] mean the student isn't as active socially on campus and is more likely to drop out," he suggested. "Sensors in the classroom could give us more data about student attendance and how they are best using it. [Or] you could use clicker data to add to your predictive and prescriptive models."

The use of alternative data sources, he added, is still "an emerging field and has a lot of potential." But the Family Educational Rights and Privacy Act (FERPA) calls for "having a good understanding of what is permissible and what is not."

6) Put Data Governance in Place First

Pursuing a good understanding of the issues and ethics related to student privacy leads to one of the most important foundations for undertaking the work of data analytics: the need for a university-wide data governance strategy. "We need to understand how we handle data in the organization," Bukralia declared. That encompasses multiple aspects: agreement that data is a strategic asset; that data silos across campus need to be broken down; and that an information architecture is required in order to understand how the data flows in an organization.

7) Prescriptive Analytics is a Project to be Managed

Just as with any campus project, Bukralia explained, a prescriptive analytics endeavor requires gaining buy-in from all stakeholders: IT, the business unit sponsoring the project, top leadership, students, faculty and those doing the work. It requires a business sponsor backing the project. Those who sponsor the project need to participate in defining acceptable key performance indicators for monitoring the work, as well as goals for measuring the outcomes or benefits of undertaking the project. And resources need to be identified, both human and data: Does the institution have the data it needs in order to make insightful decisions? Is it usable? Is the quality of the data up to the standards defined by the data governance strategy? After the project has been delivered, a feedback loop should be in place to assess what went well and what needs improvement.

8) Skills May be Lacking

The people aspects of analytics can't be underestimated, according to Bukralia. As an emerging field, data science has a "dearth of people who have a good understanding of analytics," he noted. The result -- particularly for smaller institutions -- is that finding high-quality analytics and data scientists could prove to be a major challenge. Don't assume the resources will be there when you need them unless you put a lot of energy into recruiting and training.

9) Prepare for Change Management

Analytics projects don't always produce the results expected. When results fall short of expectations, the institution may need to modify its goals or change how work gets done. But like any organization, schools have varying appetites for change. "The bottom line is that if you have a wonderful analytics model, and as a result you have to implement some new ideas [that] are backed by a lot of data and cross-validated, but then there is no appetite in the organizational culture for change, that will lead to conflicts," Bukralia pointed out. "The question is, will you be willing to implement actions suggested by prescriptive or predictive analytics where they intersect with the culture?"

10) A Model That Works for One Institution Won't Necessarily Work for Another

While best practices and steps for developing a prescriptive model are broadly applicable, colleges and universities are typically too unique to be able to use models developed elsewhere. As an example, "an institution where you are seeing a lot of diversity will have different kinds of constructs and variables that you will need to take into account in building a model of high predictive accuracy compared to an institution where those factors related to diversity don't matter much," Bukralia explained." Analytics solutions by and large are not like one-size-fits-all kinds of solutions."

11) Benefits of Prescriptive Analytics Come in Two Flavors

Institutions should expect both tangible and intangible benefits of prescriptive analytics. Tangible ones might include measurable improvements in retention rates, which could translate into monetary terms. Intangible benefits might be improvements in "cycle time to action" said Bukralia: for example, where better decisions can be made and made more quickly, where resources can be pulled off of certain types of work and redeployed to other higher-value work or "where students are happier with their learning experience and parents feel better about the institution also."

12) Prescriptive Analytics Aren't Always Right

While Bukralia believes that the use of predictive and prescriptive models holds great promise, statistical modeling still has its blind side. "You have to be able to account for sudden changes in human behavior," he stressed. "You may have a student who has been topnotch all along but there are some changes that happen in that student's life, and suddenly that student decides to drop out. That would be a sudden change. I would call it a 'mutation' -- a major change in someone's behavior that's hard to predict," he explained. "Statistical models are not very good at handling unpredictable human behavior."