Research

Online Ed Leaders Agree Top 2 Indicators of Program Quality Are Student Success Rates, Student Evaluations

• By Rhea Kelly
• 04/02/18

The majority of online education leaders agree that most relied-upon metrics currently in use by online programs are student retention and graduation rates and student course/program evaluations, according to a recent survey from Quality Matters and Eduventures Research. While those two indicators topped the list by far, other quality measures, such as faculty training, summative student assessment, program reputation and external rankings, were less commonly cited by the survey respondents.

The second annual Changing Landscape of Online Education (CHLOE 2) report surveyed 182 chief online officers (COOs) at U.S. colleges and universities about policies, practices and plans around online education. Researchers defined "chief online officer" as any position that manages online education for an institution — with responsibilities spanning course and program development, training, technology selection, support and oversight, budgeting, quality assurance, planning and policy.

Researchers also asked what quality indicators should be used to assess online programs in the future. Again, student success rates and student evaluations were most commonly cited by respondents. Yet two other measures were not far behind on online education leaders' wish lists: post-graduation placement/employment in related fields and program reputation. "The added emphasis on post-graduation employment may help to satisfy stakeholder demands for measurable outcomes that validate the return on investment in higher education by individuals and society," the report noted.

It could be that institutions rely on similar quality indicators because those are the metrics they can easily access. While the majority of respondents said their institutions have "adequate data to demonstrate performance" on indicators such as faculty credentials, retention/graduation rates, cost of attendance, student evaluations and faculty training, fewer were able to measure data such as student achievement of program objectives, expenditures per student, and standardized test performance. And more than half of institutions lacked adequate data on loan repayment statistics, student debt, external certifications, post-graduation employment, graduate earning, employer feedback and alumni feedback. The report noted that the types of indicators where institutions have insufficient data tend to be information on student outcomes.

The top three reasons for this lack of outcomes data, according to the survey respondents: insufficient resources to collect the data, difficulty of obtaining existing data, and lack of established benchmarks.

The full report, which explores trends in the management of online education as well as the tools and techniques employed in online programs, is available on the Quality Matters site (registration required).