A PAR Update and Vision for Predictive Analytics in the Transformation of Education
Q&A with Ellen Wagner
"We need to keep our eyes on our true values as education stakeholders and remember why we 'do' education at all." — Ellen Wagner
It's been two years since Hobsons acquired the Predictive Analytics Reporting Framework (PAR) in January 2016. CT has been following PAR's development as an important model for higher education, from its inception eight years ago as a research project of several higher education institutions, to its current home at Hobsons.
Education visionary Ellen Wagner — now VP for Research at Hobsons — has guided PAR's entire journey and is the best voice to offer perspectives on its significance for higher education. She has seen the project move from a collegial inquiry with her peers — both before and after PAR received its initial funding — to its maturity in mainstream product integrations and accepted education methodologies.
Here, Wagner gives us an update on PAR, as well as her vision of more general directions in predictive analytics for education.
Mary Grush: PAR has been a key foundational participant in the analytics movement in higher education. I'd like to get your update and perspectives on PAR today. So much has happened — where should we begin?
Ellen Wagner: Who would have thought that a small group of us, talking about this emerging idea of predictive analytics after the 2010 WCET meeting, would have had the opportunity to take that idea from what we thought might be "fun" to try in education, to the designing, funding, developing, and releasing of a commercial software product. Beyond that, who would have imagined our nonprofit research project being acquired by a commercial software provider? It's been quite an experience.
PAR has literally "grown up" as a part of the analytics movement in U.S. postsecondary education. Some might even say that we helped shape some of the expectations that education customers have of ed tech companies when working with analytical solutions, especially for part-time, working, and adult students — the new majority — and for students who are completing degrees in blended and online programs.
How about if we start with an update on PAR at Hobsons, and then let's take a look more broadly at the transformative impact that analytics are bringing to education practice whether we as educators are ready or not. I've got a few thoughts on what those of us who care about student success need to be thinking about in 2018.
Grush: Great! As a set of methodologies and common definitions for predictive analytics, PAR has been integrated into Hobsons products for student success. Could you tell me what some of these applications are and how PAR fits into Hobsons's student success business?
Wagner: Product integrations are a tricky business. As much as it would be awesome to connect a few lines of code and have lots of magic happen, true product integrations often require reworking a lot of product architecture as well as underlying code and data sources, and doing so without any interruptions or changes to functionality. I mention this because we are not "just" integrating products because we can. We are looking into bringing the power of analytics to college and career readiness planning, and to college matching and fit, just as we have to risk mitigation and to broadening our footprint in intervention efficacy.
Our primary work to date has focused on integrating PAR with our Starfish platform, since that is where the most immediate opportunities for post-secondary student success improvement can be found. PAR analytics are now tied to "smarter" advising and case management and are increasingly being used with interventions inventory, and efficacy assessment. As we have seen from our PAR research, college fit is one of the most predictive indicators of long-term student success; my colleague at Hobsons, Dr. Susan Hallenbeck, advised us that fit is a strong predictor of retention and completion. You can be sure we are spending some quality time looking beyond predictions for matching success, to focus on the importance of finding what has come to be known as 'fit': the feeling of belonging, of knowing that one has found a home base.
Additionally, our data analysts are bringing predictive analytics to Naviance, our college and career readiness platform. We need to be smart about where predictions of readiness make the most sense. Clearly if we are concerned about risk in college, of course we are also concerned about readiness in middle school and high school. Why not extend the PAR Framework to addresses college and career readiness, and college matching and fit, as well as college progression, retention, and completion?
Grush: Hobsons recently sold some of its product offerings to Campus Management to enhance that company's CampusNexus platform. The acquired tools seem more relevant for CRM platform and business management systems — these applications are less closely related to student success. By releasing those tools to Campus Management, is Hobsons now going to be focused exclusively on its student success applications?
Wagner: Hobsons made a corporate decision to focus on a value proposition of student success, articulated and demonstrated through its Naviance platform (college and career readiness), on its Intersect platform (college matching and fit), and on its Starfish platform (advising, early alerts, case management, predictive analytics, and intervention efficacy). As you can tell from my answer to your previous question, Hobsons has a "big picture" vision for where we want to go with our student success solutions — independent of the CRM platform tools that, until recently, defined our footprint in our higher education business.
Grush: Of course, once PAR was acquired by Hobsons and integrated into Hobsons product lines, PAR’s impact and relevance was felt more widely. Will PAR eventually be integrated into other tools, at other education companies beyond Hobsons? How?
Wagner: One of our strategies for extending the power and reach of our analytical capabilities is to actively seek partners with whom we can develop APIs for extending functionality. While there are some companies, and perhaps even some customers, that will want us to continue to believe in the power of the "single stack", I think many of us providing services in the land of education technology are going to turn to APIs to mix and match different kinds of ed tech solutions so that stakeholders can customize the platform services they provide to their users. This is true whether one is a commercial provider of platform services, or running a data center on a campus. Flexibility and responsiveness will continue to be key.
Grush: Over the years, PAR evolved far beyond its original objectives. Could you tell me what you think of as the highlights of strides PAR made beyond its initial concept? Have these changed or extended the original values of the PAR project?
Wagner: PAR's original objectives were literally to see if we could use predictive analytics to find students at risk of dropping out of college, which, when you think about it, continues to be a worthy goal. We still have a long way to go! Back when we hatched this idea in 2010, we wanted to see if we could identify differences among students at risk of dropping out of different kinds of schools and programs. We were tremendously excited when we saw that our predictive models were able to help us predict student risk in the areas of progress and retention. We didn't have the data then — until some students at our member institutions reached graduation — for completion predictions, but we were excited about the trends.
Moving ahead, we created common data definitions so that all of the schools in PAR could contribute their de-identified data to our single dataset. Over time this helped us with consistency and scalability. It's important to note that we focused on outcomes, not processes.
We saw our participants working together, learning to trust each other in a really new environment. Sounds simple now, but we had a lot of natural enemies in the "room where the data happened". For-profit institutions and community colleges and online schools and competency-based institutions and traditional institutions, all sharing their student data. That was huge.
As Beth Davis, who served as our CEO at PAR, and I saw our success building even during the early years, we also recognized that if we didn't know what we were going to do about getting students out of risk, then our work was never going to be particularly meaningful. That is when our interest in interventions and motivation began, which brought us to the Student Success Matrix. We went in search of advising, and early alert, and watch lists. We found Russ Little and all he had done with the Student Success Plan… Some of our members wrote EDUCAUSE IPAS grants to integrate PAR and Starfish and before you knew it — Boom! — PAR became part of Hobsons.
It seems to me that in our quest for actionability, we saw that analytics are tools we get to use to help anticipate making better decisions to support student achievement. And really, that was clearly our goal all along. So maybe what we are seeing more than anything is the value of action research, the benefits of using evidence to test our strongly held beliefs. More than anything we have learned that we should never, ever take our eyes off the prize of keeping student success at the center of our work.
Grush: The work you've done with PAR over the course of more than eight years — plus, of course, the totality of your career in education — has put you in a unique position to recognize the directions predictive analytics may take us in education and the opportunities analytics strategies offer us to influence positive transformations.
Let me frame my question this way: PAR, with its immense database and the scrutiny of its education community membership, has proven the effectiveness of predictive analytics methodologies in education. Where can we go from here — in a general sense — using predictive analytics for the benefit of education? And what do we need to be thinking about now?
Wagner: William Vorhies wrote a provocative blog post a few months ago in which he asserted that it was time for predictive analytics to take a victory lap. It's true that people no longer immediately assert that predictive analytics are the devil's business when talking about education research. Still, as recently as 2015, I was asked by graduate students in education disciplines to join their dissertation committees as an external reviewer, to make sure that at least one person on their committee viewed predictive analytics as a legitimate research methodology for a dissertation in education. Old habits die hard in the academy.
Nevertheless, the recognition that data scientists are bringing predictive analytics, machine learning, deep learning, and artificial intelligence to the table has silenced some of the loudest objections, almost to the point where education researchers now worry about losing their research voice as their methods are trumped by data science's more mathematically oriented models. This is a topic we discussed at some length at last year's Research Symposium at the Teaching and Learning at a Distance Conference in Madison.
These days, many of us in education are finding ourselves held in thrall by the sirens of deep learning and artificial intelligence. I think part of the allure is the hope that perhaps the secret to analytical success will be contained in increasingly complex and sophisticated findings — more than in what has been uncovered to date. It's sort of like looking for data DNA. I suspect this also speaks to the rapidly emerging field of learning engineering, but that is an entirely different conversation.
What all this is really pointing to, to me, is that we are pretty much done with the so-called 'awareness phase' of predictive analytics. These days we find ourselves in the midst of exploration and adoption, whether we like it or not. With that comes the first wave of education transformations from having insight and information about how our practices affect student and faculty performances, processes, and learning outcomes.
One of the things we know about change in education is that we love to talk about it. Another thing we know about change is that it is really hard to do. As Dilbert once noted, "Change is good. You go first." For a real transformation to succeed we first need to understand the problems we want to solve and the opportunities we want to realize.
Next, we need a plan, with clearly stated goals and the commitment to see the plan through to its implementation. We need to know: How do these technologies make a difference? How much? For whom? When? Without that, we've got nothing more than new technology, with little opportunity to show whether it is working or not — not exactly the transformation your provost is going to be looking for at the end of the year.
Then finally, we can put that plan to work and drive true transformational value. But we need to stay grounded: We need to keep our eyes on our true values as education stakeholders and remember why we 'do' education at all.
More than ever before, I know that student success is not software, and that predictive analytics is not punitive if used ethically and responsibly. Student success is the shared value construct of helping our citizens achieve their dreams of a better life for themselves and their families. Any of us who care about the future of our communities and our world should be able to get behind that.