AI Case Study
Georgia State University improved on-time enrolment by 3.3 percentage points using a machine learning chatbot to guide students during the process
Georgia State University trialled a chatbot to address incoming students' specific concerns and ensure they were on track with university onboarding. This reduced summertime "melt" - wherein students accepted to university do not end up enrolling in the fall -by 3.3 percentage points.
Industry
Public And Social Sector
Education And Academia
Project Overview
From Harvard Business Review: "In collaboration with Georgia State University (GSU), we tested whether “Pounce,” a conversational AI system built by AdmitHub and named for the GSU mascot, could efficiently support would-be college freshmen with their transition to college. Pounce features two key innovations. First, the system integrates university data on students’ progress with required pre-matriculation tasks. Thus, rather than providing generic suggestions, Pounce matches the text-based outreach that students receive to the tasks on which data indicates they need to make progress and therefore may need help... In this way, the system provides students with individualized outreach. Second, the Pounce system leverages artificial intelligence to handle an ever-growing set of student issues, challenges, and questions (e.g., When is orientation? Can I have a car on campus? Where do I find a work-study job?). The system can be accessed by students on their own schedule 24/7. It can efficiently scale to reach large numbers of students, and it gets smarter over time."
Furthermore, with regards to implementation costs from the research paper: "An important question relates to the cost of this intervention in comparison to previous summer melt intervention. The cost of the AdmitHub platform ranges between $7 and $15 per student per year in addition to per student costs associated with staff involvement in establishing the messaging system and monitoring student communication not handled automatically.16 These costs are less than prior summer melt interventions involving individual counselor outreach, which ranged from $100 to $200 per student (Castleman, Page, & Schooley, 2014) and on par with those involving non–AI based text-based communication (e.g., Castleman & Page, 2015)."
Reported Results
According to the research paper, "Approximately 85% of treatment students responded to the Pounce system at least once. Despite the high rate of engagement, only a small share of students (13.5%) sent messages that the system could not handle automatically and therefore routed the messages, via email, to a GSU staff member.
GSU-committed students assigned to treatment exhibited greater success with pre-enrollment requirements and were 3.3 percentage points more likely to enroll on time. Enrollment impacts are comparable to those in prior interventions but with substantially reduced burden on university staff. "
Technology
From the research paper: "Through a process of supervised machine learning, AdmitHub trained the Pounce system by observing all student conversation logs. The practical process of training conversational AI requires humans to review all chat logs and manually identify instances of error or confusion. That human feedback is then incorporated into the ongoing training process. This process is referred to as deep learning with convolutional neural networks."
Function
Customer Service
Other
Background
From Harvard Business Review: "From the perspective of an AI system, the college transition provides intriguing challenges and opportunities. A successful system must cope with individual idiosyncrasies and varied needs. For instance, after acceptance into college, students must navigate a host of well-defined but challenging tasks: completing financial aid applications, submitting a final high school transcript, obtaining immunizations, accepting student loans, and paying tuition, among others. Fail to support students on some of these tasks and many of them — particularly those from low-income backgrounds or those who would be the first in their families to attend college — may succumb to summer melt, the phenomenon where students who intend to go to college fail to matriculate. At the same time, providing generic outreach to all students — including those who have already completed these tasks or feel confident that they know what they need to do — risks alienating a subset of students."
Benefits
Data
From the research paper: "By integrating data from the university’s student information and customer relationship management systems, Pounce could send students messages that were personalized to students’ immediate needs for those domains where they were failing to make progress or raised questions. For example, only students who had yet to file the FAFSA would receive FAFSA-related outreach. To automate responses to student questions, GSU admissions counselors seeded a knowledge base with approximately 250 frequently asked questions. Over the course of the intervention, this knowledge base grew to more than a thousand as the system learned through engagement with students.
3,745 students were assigned to received Pounce outreach, and 3,744 students were assigned to the control condition."