Research

Nudging Works in CCs Among STEM Students

• By Dian Schaffhauser
• 09/10/19

Does nudging students at key points in their college careers work or not? A recent working paper published by the National Bureau of Economics suggested that nudging had no impact on massive groups of students. Yet, two organizations working to instill innovation in American education found big success in "nudging" strategies for STEM students within community colleges.

In a project undertaken by nonprofit JFF and for-profit ed tech company Persistence Plus, text nudges increased first-to-second-year persistence for STEM students by 10 percentage points and exposed particular success among students of color and those over the age of 25. The multi-year research involved about 9,500 students at four community colleges.

"Nudging to STEM Success," as the program was called, involved three schools in Ohio — Lakeland Community College, Lorain County CC and Stark State College — as well as John Tyler CC in Virginia.

The experiment started with a randomized control trial covering 2,700 students during summer 2017. That pilot project found that nudges resulted in a 10 percentage point increase in persistence for STEM students compared to a control group.

People at each campus worked with behavioral science and technology experts from Persistence Plus to develop a mobile text-messaging system that allowed students to receive encouraging automated responses and to ask questions. At Lorain County CC, for example, administrators used the nudging to help students address basic needs and overcome barriers to success, directing them to services such as financial planning, tutoring and advising programs, as well as the college's on-campus food pantry.

As Lorain County CC President Marcia Ballinger pointed out, in a statement, "Often, the barriers that today's students face are complex and multifaceted — balancing work, family, transportation, health and wellness and finances. To overcome these challenges, students need encouragement. They need to know they are not alone and that there are resources available to help them keep going under adverse circumstances."

During the full implementation of the experiment, nudges increased persistence by 16 percentage points: 72 percent for subscribing students versus 56 percent for those who chose not to be nudged. Among students of color, 62 percent of those who opted in persisted after their first semester, compared to 46 percent who opted out. And for students over the age of 25, the nudging showed even greater impact: Sixty-four percent of those who chose nudging persisted after their first semester, versus 44 percent of those who opted out.

According to a report on the results, in an examination of continuous enrollment over three terms, from fall 2017 through fall 2018, persistence remained high for those choosing to be nudged:

• Two-thirds of students (65 percent) who remained subscribed persisted, compared to slightly over half (52 percent) of those who unsubscribed from the nudges;
• A similar number of students (64 percent) over 25 who were subscribed to nudges stayed enrolled over three terms, versus to 52 percent of unsubscribed students; and
• Nearly six in 10 (58 percent) of subscribed students of color remained continuously enrolled across the three semesters, versus 38 percent of unsubscribed students.

The researchers emphasized that there were "some limitations" in using this method of measuring. Those who choose not to engage in support services "may differ" from those who are willing to take help. And those who opted out from the nudging may have simply chosen not to continue in college.

Still, the report noted, "these outcomes suggest that nudges have a meaningful impact on student persistence at scale."

So, why did the NBER study come up with different results regarding the effectiveness of nudging? One point of that study was to examine the scalability of nudging. The JFF/Persistence Plus study was much smaller (10,000 students compared to 800,000). Also, noted Cecilia Le, managing director for Persistence Plus, the approaches were different. "We developed and designed personalized nudges to support students holistically. Our nudges addressed challenges ranging from food insecurity to the psychological barriers that lead students, especially underrepresented students, to withdraw from STEM, and we collaborated with our college partners to customize the messaging for their particular campus context."

The researchers in the JFF/Persistence Plus study found that students they interviewed for the study reported that the just-in-time nature of the messages were helpful in "motivating them to keep going through difficult times" and remembering "what they needed to do" for classes and processes such as registering and filling out financial aid applications. As one student told them, the nudging "gave me motivation when I wanted to quit."

"These results send an encouraging signal to institutions working to identify scalable strategies to improve persistence, and completion, for students of color within STEM fields," said Jill Frankfort, president of Persistence Plus. "We believe these additional findings and insights will help to advance the field's understanding of nudging — and its potential to improve student success."

The Ohio colleges plan to continue using the nudging strategies and have expanded them beyond STEM programs. The fourth college, John Tyler CC, has become engaged in another initiative focusing on students near the end of their degree paths, but expects to move forward on the lessons it learned regarding how to communicate with students.

The results of the project will be presented in an upcoming peer-reviewed paper titled "A Summer Nudge Campaign to Motivate Community College STEM Students to Reenroll." The paper will be published in AERA Open, an open-access journal published by the American Educational Research Association.

The research was funded by the Helmsley Charitable Trust.

The full report of findings is openly available on the JFF website.