Data Analytics
9 Ways to Doom Your Data Analytics Efforts
While data has become an essential tool for decision-making on campus, analytics are a tricky business. Here are nine pitfalls to avoid.
Shutterstock.com
Higher education has been talking about big data for years now. And most schools have come around to the idea that massive amounts of data can be collected, analyzed and used to make recommendations to decision-makers who must determine the best paths for their institutions.
Yet, as a practical matter, data analysis and data-driven decision-making are still in their infancy. According to the Education Advisory Board, just eight institutions have a staff member who carries the title of chief data officer — out of 4,000 colleges and universities in the United States.
It's no wonder, really. There are countless pitfalls and obstacles on the way to successful data analytics. Here are nine common mistakes, and what some institutions are doing to avoid them.
Mistake #1: Reinvent the wheel.
Deciding to move to data-based decision-making is a significant step for any institution to make. It will take time, resources and work. But you don't have to reinvent the wheel, as they say. Every department and business unit in your organization has been collecting data for decades.
Start by finding out what you already know. It might be good news: Maybe you already have enough information to start answering some simple questions. Or, you may learn just how redundant some of your data collection is. And you'll certainly learn what you don't know.
"The simplest first step is to take a look at what you already have and begin to mine it using fairly traditional techniques," said Jim Kulich, vice president and chief information officer at Elmhurst College in Illinois.
Mistake #2: Aim too high.
You don't have to transform your entire institution with your first data-based initiative. When you start out, keep it simple. Decide what you want to know, what decisions you want to inform.
"You can ask yourself some simple questions to begin," said Kamran Khan, vice provost for information technology at Rice University (TX). "Maybe you want to know how to predict student success. How do you look at different types of courses being taught? What are your space requirements?"
No matter what, start with some very specific questions to answer and see where that takes you.
Mistake #3: Assume everybody's talking about the same thing.
How many students do you have? Would the answer to that question be the same if you asked the registrar or the dean of academic affairs? Do you count everybody who takes a single class or just those who have full schedules? And when do you count them? At the beginning of the semester? Halfway through? At the end?
This is where the concept of data governance comes in. You need to come up with a mechanism so that everybody on campus knows they're talking about the same thing. "It's important to make sure you have a good data governance group from different areas across the campus," noted Khan.
Michael Chapple, senior director of IT service delivery at the University of Notre Dame (IN), has had a full-time data steward on staff for a year and a half, but Notre Dame started its data governance project a few years before that. At this point, the university has more than 800 different definitions it uses when dealing with data.
"When you start pulling information into a system, all of a sudden it really does matter that we all have the same idea of what we're talking about when we say the word 'student,'" Chapple said.
Mistake #4: Rely on averages.
Slice and dice to your heart's content, and you can probably come up with an average or mean to describe anything. But what difference does it make?
"For example, if you look at any common data source, you'll get an unholy term called the average discount rate," Kulich said, "which is nothing more than the percentage of your tuition income that you rebate to financial aid."
If you know that, what do you know?
"Maybe you've got super-smart kids who have high ACT scores that you're giving big scholarships too," he said, "but maybe you've got other kids who aren't so stellar academically but their families have means. Nobody is average."
When you examine data, make sure you know what the distribution looks like. Get a sense of the shape of things before you start making recommendations that could lead to bad policy.
Mistake #5: Believe you know what you're doing.
You have people on your staff who have been engrossed in collecting data and spitting out reports for years. You can trust them to take the next step and start analyzing that data, right?
Maybe, maybe not. Academia has been working with big data and data analytics for years now, and very smart people are coming up with new methodologies all the time. Doesn't it make sense to take advantage of those new ways of thinking about data?
"You don't just go and find the person who did this 15 years ago for a master's thesis using some limited methods and apply them again," Kulich said. "There are simple things that are new and can make a big difference."
Mistake #6: Choose security over transparency.
Those who either officially or unofficially have been referred to as "data stewards" have long taken their responsibilities seriously. What's more, there are federal and state laws that compel institutions to protect data security and privacy.
"We come from environments where, unless you had a need to know the data, it wasn't naturally shared and we weren't transparent about it," said Lisa Davis, chief information officer at Georgetown University (DC).
And yet, if institutions of higher education want to take better advantage of the data it collects to make intelligent decisions, they have to find a way to share it.
"How do we make the data accessible but also protect it at the same time?" Davis asked. "Finding the right balance between those two is really important."
Mistake #7: Neglect institutional leadership.
Being successful with any data-driven decision-making initiative has to mean you have the full support of the institution in collecting data, analyzing it and then incorporating it into recommendations that create change.
"You're going to need to rally the troops around the institution," Chapple said.
And you can't do that unless you have the full support of the top leaders in your institution. You need them to believe there's value in what you're doing, take it seriously and then communicate that belief to everybody else on campus.
"Absolutely the most critical thing is to get leadership support for the effort," he said. "If you don't have senior-level support, you can't hope to be successful."
Mistake #8: Give your president the whole picture.
For even the smartest, most analytical decision-makers, a blur of numbers is … a blur of numbers. Give somebody too much information that does not appear to be relevant, and you're not really helping them very much.
If the data tells a story — as those who love data believe it does — find a way to tell the story.
"Data visualization is a huge piece of data analytics," said Mike Kelly, chief data officer at the University of South Carolina. "You have to have a grasp of statistics, but there's also psychology. You've got to understand what types of visuals make things real for somebody."
Mistake #9: Think this is going to be quick and easy.
Maybe you do start out with some simple questions, but the hope, of course, is that you can apply some sweeping transformations that lead to student success and revolutionize your institution. Never assume that applying a simple formula to a mass of data is all it will take to change the course of a complex organization.
"This kind of thing definitely takes a lot of time and work," Khan said. "You need to keep people up-to-date on where you are and you have to have buy-in from the community."
