Effects of Voice-Based Artificial Intelligence (AI) in Customer Service: Evidence from a Natural Experiment

Voice-based artificial intelligence (AI) systems have been deployed gradually to replace traditional interactive voice response (IVR) systems in call center customer service, but little evidence exists on how the implementation of AI systems impacts customer behavior, as well as AI systems’ effects on call center customer service performance. Leveraging the proprietary data from a natural field experiment, we examine how the introduction of voice-based AI affects call length, customers’ demand for human service, and customer complaints in the call center customer service of a large telecommunication service firm. We find that the implementation of the AI system significantly increases call length and decreases customer complaints. Although the AI-based service system presumably reduces users’ efforts to transfer to human agents, we do not find any significant increase in customers’ demand for human service. Furthermore, our results show interesting heterogeneity in the effectiveness of the AI-based service system. For simple service requests, the AI-based service system reduces customer complaints for both experienced and inexperienced customers. For relatively complex quests, customers learn from prior experience of interacting with the AI system, and this learning effect leads to fewer complaints. Moreover, the AI-based system exerts a significantly larger effect on reducing customer complaints for older and female customers, as well as for customers who are experienced in using the IVR system. Finally, in examining details in customer-AI conversations, we find that speech-recognition failures in customer-AI interactions result in an increase in customers’ demand for human service and customer complaints.