Many Student Success Initiatives Have No Impact on Retention

• By Rhea Kelly
• 09/10/19

According to a recent analysis from Civitas Learning, only 60 percent of student success initiatives have a measurable positive impact on student persistence. That leaves a full 40 percent of initiatives having no impact whatsoever.

For its 2018-19 report on the state of student success initiatives, Civitas Learning looked at the impact of about 1,000 initiatives at more than 55 colleges and universities, including Greek life, advising, first-year seminar programs and other services, to find out what actually works best for students. Among the findings:

• Within that 60 percent of student success initiatives demonstrating a positive impact, 20 percent still did not work for certain student groups.
• While 40 percent of the analyzed initiatives had no impact on retention overall, 15 percent of the analyzed student segments actually did experience better outcomes.
• The top five most effective student success initiatives were: adviser meetings (producing a 5.8 percentage point increase in student persistence), Greek life (3.8 percentage points), supplemental instruction (3.4 percentage points), scholarships (3.2 percentage points) and tutoring (3.0 percentage points).
• The effectiveness of student success initiatives varied by student subgroup. For example, while Black and Hispanic students benefited most from a mix of personalized academic support, White students benefited from a mix of academic, financial support and student life programs.
• Academic and financial support were particularly effective for first-term students.
• The impact of student success initiatives generally lessens over time, but scholarships, adviser meetings, career planning, tutoring and supplemental instruction all remain effective over the long term.

The report also addressed the current limitations of student success data analysis: "Not only is there a limit to the analytical prowess of some [institutions], but the democratization of data (and consequently, front-line access to the data they need to inform pinpointed, student facing change) is a challenging task for any institution to tackle. Without the ability to understand what could be complex analyses, some data could be used in inaccurate or even harmful ways."

Two essentials for better analysis, the report asserted, are that: 1) "Both senior leaders and student-facing employees need to have access to data that serves their specific needs," and 2) "Data and conclusions drawn from data need to be accurate and actionable in a meaningful, concise, and obvious way." And to achieve both, the report offered three next steps for institutions:

• Strive to allocate student success funding with pinpoint precision;
• Conduct an honest and objective self-reflection of who comprises the student body, what services they are provided and what they need; and
• Address analytical limitations with improved mechanisms for analysis and guided interpretation and implementation of results.

The full report, "What Really Works: A Review of Student Success Initiatives," is freely available on the Civitas Learning site.