Q&A
Why Data Is the Most Important Tool for a Higher Education Leader
Tech Tactics in Education speaker Jason Simon shares how colleges and universities can better utilize data to tackle existential challenges and move toward analytic maturity across the institution.
As associate vice president of data, analytics, and institutional research at the University of North Texas, Jason Simon believes in the transformational power of data to improve student and institutional outcomes. This November at the 2023 Tech Tactics in Education conference in Orlando, he will outline how analytic maturity can lay the foundation for higher education institutions to thrive in the "new normal" in his session, "Higher Ed Is in Trouble: How Analytics, Data Literacy, and Data Governance Can Help Save It." We caught up with him to find out what holds institutions back from making use of their data, what the road to analytic maturity looks like, how AI will impact how institutions approach analytics, and more.
Campus Technology: The title of your session posits that "Higher Ed Is in Trouble." Why?
Jason Simon: Part of it is the reality of shifting demographics: We're seeing shrinking numbers of high school graduates in the vast majority of the United States. There are only three states that are seeing any growth whatsoever. And the mergers, the acquisitions, the campus closures — including some very large historic institutions — all point to the need for institutions of higher education and their leaders to think differently about how they leverage data, analytic maturity, data governance, data literacy, and really start to take advantage of that to help address some of these challenges.
Another factor is the overall student belief in higher education being worth the investment. A recent Gallup poll found that American confidence in higher education is at an all-time low — nearly a 20 percentage point drop compared to eight years ago. There's a burden for institutions of higher education to demonstrate to students, their families, and legislatures the value that they provide in return for the tuition and fees that they charge. Many institutions can't do that because they lack the maturity on connecting their student data with their finance data, with their individual lifetime earnings data, and they're stuck trying to tell a story with major plot lines missing.
CT: What are some common things that are holding institutions back when it comes to using their data?
Simon: The biggest piece that really holds campuses back is not having a clear, articulated executive sponsor behind their data efforts. These initiatives require collaboration across the enterprise to really marshal resources, staffing, and prioritization of activities to improve a data ecosystem.
Second, most campuses haven't done a thorough data audit. They haven't done a data maturity audit. They haven't engaged stakeholders widely and broadly across the institution to understand the complexities of the problems that they're up against.
Data literacy is also a common challenge. Data is, by all accounts, one of the most important tools that we have as an administrator and a leader for institutions of higher education. If it's non-existent, or if it's stuck behind a narrow stovepipe that's not shared very well, or it's governed in such a way where it's not federated and democratized across the institution, then it's really of no value.
Let's assume that an institution has figured all that out. They have a mature data system, they have wide-scale culture of adoption. Then the burden really shifts from the data provider to the data consumer. And it's hard to get a leader or a decision-maker to move from what I like to call "me-search," which is what they think and believe and perceive, versus research, which is actual data-informed decision-making. The other challenge is, does the institution have the right level of analytic maturity to look beyond basic descriptive statistics and reporting?
Much of the work being done by AIR, Educause, and NACUBO, first with the Joint Statement on Analytics, and now with the Bill and Melinda Gates Foundation grant that they recently received, is driven at trying to provide tools for institutions of higher education — all the way from two-year to four-year, public, private, doesn't matter — on how to actually move the needle in getting a more mature data ecosystem.
CT: You mentioned many institutions have not performed a data audit. What does that entail, and what's keeping institutions from doing it?
Simon: Most of the time it's not for any nefarious reason. It's just because the requests that data provisioners are fielding are at such a high volume that they don't have the luxury of time or prioritization or someone guiding them to do it.
Most modern data provisioning shops in higher education right now are consumed by federal reporting, state reporting, ad hoc requests — and they lose sight of the opportunities that analytics can provide them in terms of self-service data. For us at the University of North Texas, before we could even get to a data audit, we first had to really understand who are our data partners.
The first step is to develop a RACI matrix by data typology across the campus. RACI is who's Responsible, who's Accountable, who needs to be Consulted, and who needs to be Informed. It's a common practice in a lot of IT areas, but in terms of a data audit, it's vital because it sets the stage on understanding who you're going partner with across the campus on what type of data, be it financial aid, human resources, finance, student accounting. It also serves as a necessary first step for data governance. In essence, you're identifying key data governance partners, both functional and technical. And lastly, that RACI process helps the data leader or chief data officer pinpoint who they need to build strategic relationships with and how to maximize those relationships.
Once you know who your partners are, it's about understanding your institution's data landscape. There are several ways to do that. One is to bring together all those people and have a very honest and transparent conversation on what's working with our data and what's not, where are areas of opportunity, where are areas of weakness, and what's going to be the plan to address those. It could also be a series of focus groups between the data shop on campus and key executive stakeholders. Ask very direct questions: not just how do you feel about data, but when's the last time you leveraged data to make an informed decision? When is the last time you really needed data and the institution wasn't able to provide it? If you could wave a magic wand, what data would you like to see arrive in your inbox every day? It's also important to engage with folks who are "data-adjacent": your IT staffs, your information security teams, your key people who are in the trenches with their hands on the data on a daily basis, in key higher education systems like enrollment, finance, human resources. All of that leads organizations and institutions to identify pretty quickly where their pain points are, and then hopefully with a good culture of leadership, determine how to begin to address those.
CT: When you're working toward analytic maturity as an institution, is there an end goal that is achievable and you're done? Or is it more of a process of continuous improvement?
Simon: There is a continuum of maturity that we like to strive for, based on the work of Tom Davenport, who is considered the father of modern-day analytics. At the very lowest end of the spectrum, it's hand-entered information, hand-scraped, retyped, etc. That's even before the reporting. The next phase is your basic regurgitation of what we already know — autopsy data. It already happened. You can't fix it. Then we move up, in terms of maturity, to alerting: We think something's about to happen, or we might have observed something just happen, and we have an opportunity in the short term to leverage data to begin to address it and intervene. Then we get into predictive analytics — instead of thinking about what happened in the past, we try to determine what is the best that could happen in the future. That's an inflection point where I see a lot of institutions of higher education begin to turn the corner on addressing some of these real big challenges that are ahead of them. They're taking their destiny into their own hands and they're running simulations of "what if" predictions and really trying to change policies, practices, and procedures related to that. Then lastly, machine learning and artificial intelligence are at the upper end of that spectrum of maturity.
ML and AI is an area where institutions with smaller staffs or smaller budgets might begin to leverage those practices a little differently to enable targeted interventions or focused support systems for students. But I would say most institutions of higher education are not there yet. The majority of institutional leaders who I talk with or call me for guidance, they're still either in the alerting phase or just barely starting on predictive analytics.
CT: How do you think emerging tools such as generative AI are going to impact how institutions approach analytics? Could AI help institutions move toward their analytics goals faster?
Simon: I like to call myself a healthy skeptic or a supporter with an asterisk. I'm very much focused on how natural language processing tools like ChatGPT are moving us ahead in the space. But I would say that many of these technologies are still transactional. They are good at regurgitating information, but not necessarily synthesizing it and using storytelling to help an administrator at a particular institution really do something with the data.
For example, ChatGPT can help individuals scour historic data in institutional fact books, and identify fairly quickly what that data is and what the trends might be, but it's not necessarily going to help with more complex questions. What policies and procedures would we need to change in order to see a 2% increase in retention and what impact would that have on our net tuition and revenue? That's not something that ChatGPT can answer in its current form. Someday, yes, it should.
At the same time, I've seen ChatGPT be incredibly useful to institutional researchers who are looking for ways to bring a different level of persuasion to our work. We can utilize visual NLP tools like Adobe Firefly or Midjourney, because a picture is worth a thousand words, right? For instance, if we have qualitative research with quotes from students at our own institution, we can utilize those tools to provide a face to the words while protecting the anonymity of the actual student.
I believe we need to teach data professionals about the technology — but just like a hammer could either be used to build a home or be turned into a weapon to do harm, it's still the hammer. It really comes down to how we as professionals in technology and data are trained in how to use it. AI and NLP can help in an institution's journey toward analytic maturity, but only when the institutional culture and those who are leading the data areas have an interest, have the time, and have the desire to learn more about the technology and apply it to higher ed.
CT: What do you hope people will take away from your session?
Simon: I hope that they learn how to leverage the culture of their institution, the partners that they can build across their campus, and the obstacles that they can break down, to achieve a higher state of analytic maturity.
Attendees will walk away with a potential roadmap on how to begin to have these conversations across their institution. Because at the end of the day, data's just data. For me — I'm a first-generation college student — and really it's not about the data. It's about the decisions and the policies and the practices and the culture that can either help students get through and graduate quicker with less debt, or continue some of our practices that might make higher education more expensive or prolong students' time to graduate.
I'm hoping attendees will see the connection between the challenges that are ahead of us in higher education and the need to act now, grab this by the horn, and really champion the data conversation. Even if you're not the primary decision-maker at your institution, I believe everyone has an opportunity to impact the state of their data culture, and I hope that the session gives attendees a key push on the first one, two, three short-term things that they can do to make it better.
To hear more from Jason Simon, register for Tech Tactics in Education at TechTacticsInEducation.com.
