Data Analytics Track Student Success

Can schools do a better job of identifying and retaining at-risk students? According to conventional wisdom, a student with a low GPA is more likely to drop out, but are there other, stronger signals that would allow schools to pursue earlier intervention strategies?

Those were some of the questions the University of Tennessee, Knoxville, recently addressed with the help of students from the Haslam College of Business’s master’s in business analytics (MSBA) program. Between 2007 and 2017, the university’s first-to-second-year retention rate had hovered around 85 percent. That statistic is an important one, since the number of freshmen who return as sophomores is a metric that is heavily weighted in the national rankings. First-year retention also directly affects student enrollments, and enrollments impact tuition revenues—a major consideration in the U.S. when state funding for higher education is declining.

To gain insight into reasons why some freshmen do not return for their second year, the Office of the Provost and the Haslam MSBA program initiated the analytics project in May 2017. The provost’s office hired two incoming MSBA students who had undergraduate business analytics degrees from Haslam; those students spent the summer analyzing three years of student success data. Their task was to use their analytics expertise to identify a subpopulation of freshmen at risk for dropping out. The project continued through the fall and spring semesters as the provost’s office and Haslam funded two new research assistants from the MSBA program.

The students found that retention odds start to slip earlier than previously thought. In fact, the possibility of dropout increases after students are unable to complete just one course—after they withdraw (W) or take a no-credit (NC) grade in one of the gateway freshmen courses, such as math, English, or a foreign language. The probability of dropping out increases with every uptick in the total number of credit hours registered for but not completed during the first year of study.

The team also found that GPA is a trailing indicator—not a leading indicator—of dropout probability. This makes sense in hindsight, because W’s and NC grades are not reflected in GPAs. Since students are allowed unlimited NCs during their university careers, a freshman could receive several of these while still maintaining a good GPA. Many gateway freshman-year courses do not award credit for grades lower than a C, so students wishing to receive credit in one of these courses are required to retake it until they earn at least a C. These students could face repeated failure in their quests to earn a letter grade in these courses, and this could discourage them from continuing.

The GPA is also a trailing indicator because of how it impacts a student’s scholarship situation. After a student has attempted 24 semester hours, he or she must have a cumulative GPA that is 2.75 or higher to remain eligible for the Tennessee Hope scholarship. Students with low first-year GPAs may find the sophomore year is out of reach financially. Therefore, while a low GPA does eventually correlate with the dropout rate, the information it provides arrives too late to allow the school to pursue interventions.

And these interventions could be critical. Our project team learned that early identification of at-risk students, coupled with academic intervention, seemed to correlate with increased retention. Freshmen whose performance was poor in the fall but satisfactory the following spring were more likely to return for a second year than those who performed well in the fall but fell behind in the spring.

The student researchers made ongoing presentations to the vice provost for academic affairs, the provost, the head of the business analytics department, and the dean’s advisory council. They recommended early academic interventions that included changes to advising procedures and processes that would allow the school to identify at-risk students early in their freshman years, and these recommendations were adopted and implemented. The result was a 1.2 percent improvement in retention from fall 2017 to fall 2018.

While the improved success of at-risk students represents the most important outcome of the work, the project also saved the university as much as US$1 million in potential lost tuition.

Haslam MSBA students were asked to continue their study to identify additional segments of at-risk students. At the same time, the university is striving to improve retention rates by examining how NCs and Ws are awarded, and whether a change in that process might improve first-year student retention. Student success teams and academic advisors are working to identify at-risk students even faster so that intervention can start ever earlier—for example, during freshman advising.

At UT, we knew that the data were available to help us positively impact the university’s student retention rate—we simply had to commit to using that data constructively. As we have seen through this project, analytics can be a powerful tool for improving education.

Input for this article was provided by Robert Hinde, vice provost for academic affairs at University of Tennessee, Knoxville, and Amy Blakely of UT’s office of communications and marketing.