Using AI Chatbots to Freeze ‘Summer Melt’ in Higher Ed

• By Sri Ravipati
• 03/07/17

Students who accept offers of admission into a college or university don’t always show up for fall enrollment — a phenomenon known as “summer melt.” It’s a problem that Georgia State University (GSU) is all too familiar with: The institution’s summer melt rates have increased from 12 percent to nearly 19 percent in recent years. With traditional methods of reaching students (i.e. snail mail, e-mail and phone calls) producing feeble results, GSU decided to try another approach: smart text messaging.

According to a recent case study, GSU was well aware of the advantages of communicating with students via text messages, but was concerned about the additional workload that text messaging students would place on existing staff. So, the university partnered with AdmitHub, a Boston-based ed tech startup, to test out text-based intervention. AdmitHub works with higher ed institutions to create a virtual “campus coach” that embodies the collective knowledge and unique spirit of a school’s community. It integrates conversational artificial intelligence (AI) with human expertise to guide students to and through college.

Image Credit: AdmitHub.

AdmitHub spent the first few months of 2016 working closely with GSU’s admissions and IT teams, and in April launched the first virtual assistant in the United States to use the company's conversational AI technology. Named after GSU’s mascot, “Pounce” was introduced to half of more than 7,000 newly admitted students with a U.S. mobile phone number as part of a randomized control trial (RCT). The control group received GSU’s standard e-mail and snail mail communication, while the treatment group received reminders, relevant information, surveys and more from the chatbot via text message.

The chatbot responded to a variety of questions like:

• When is my tuition due?
• How do I send my ACT scores?
• Which parent do I use on FAFSA?
• Can I bring my salamander to Piedmont dorm?

“Every interaction was tailored to the specific student’s enrollment task,” said Scott Burke, assistant vice president of undergraduate admissions at GSU, in a statement. Burke, along with Lindsay Page, a professor at the University of Pittsburgh, and Andrew Magliozzi, CEO of AdmitHub, discussed the RCT findings at a SXSWedu conference panel, “Let’s Chat...Bots in Higher Education,” which took place Monday, March 6 in Austin. Page oversaw the trial with Hunter Gehlbach, a professor at the University of California, Santa Barbara.

Of the 50,000-plus student messages received, less than one percent required the attention of Georgia State staff

Image Credit: AdmitHub.

By the end of the fourth-month RCT, 63 percent of the 3,114 students in the treatment group had engaged with the virtual assistant on at least three separate days, exchanging 60 messages each on average. In total, Pounce exchanged nearly 200,000 messages with students in the treatment group. The university would have had to hire 10 full-time staff members to handle that volume of messaging without Pounce, according to Burke. Yet, less than 1 percent of the 50,000 student messages required the attention of GSU staff. The remaining messages were handled by Pounce or by AdmitHub staff overseeing the virtual assistant’s learning process. Notably, incoming first-generation college students — a group that frequently lacks supportive resources to navigate challenging financial, academic and social situations — and students receiving federal Pell Grants sent more text messages to Pounce than the average student user (on average 9.4 percent and 31.7 percent more, respectively).

The negative percentage for FAFSA indicates that more students had filed their FAFSA correctly and did not require additional help from GSU's financial aid office.

Image Credit: AdmitHub.

Using Pounce had effectively "frozen” summer melt: The treatment group had a 21.4 percent lower summer melt rate compared to the control group, as well as a 3.9 percent higher enrollment rate.

“As a result of the success of last season’s trial, we are not only continuing our use of Pounce this year, but also expanding its role to include several new initiatives focused on enrollment and student success,” Burke said in a statement.

“The idea is to ensure students have a positive experience with the college from first inquiry to enrollment,” said Magliozzi, in a statement. “Our technology has proven to be an affordable and extremely effective way to provide students with the near real-time feedback they expect today, while yielding positive results from the university.”

To learn more about AdmitHub’s conversational AI technology, watch the video below. Full research results are available in an AdmitHub blog post.