IDEAS home Printed from https://ideas.repec.org/a/bla/popmgt/v32y2023i4p1002-1018.html
   My bibliography  Save this article

Voice‐based AI in call center customer service: A natural field experiment

Author

Listed:
  • Lingli Wang
  • Ni Huang
  • Yili Hong
  • Luning Liu
  • Xunhua Guo
  • Guoqing Chen

Abstract

Voice‐based artificial intelligence (AI) systems have been recently deployed to replace traditional interactive voice response (IVR) systems in call center customer service. However, there is little evidence that sheds light on how the implementation of AI systems impacts customer behavior, as well as AI systems’ effects on call center customer service performance. By leveraging the proprietary data obtained from a natural field experiment in a large telecommunication company, we examine how the introduction of a voice‐based AI system affects call length, customers’ demand for human service, and customer complaints in call center customer service. We find that the implementation of the AI system temporarily increases the duration of machine service and customers’ demand for human service; however, it persistently reduces customer complaints. Furthermore, our results reveal interesting heterogeneity in the effectiveness of the voice‐based AI system. For relatively simple service requests, the AI system reduces customer complaints for both experienced and inexperienced customers. However, for complex requests, customers appear to learn from the prior experience of interacting with the AI system, which leads to fewer complaints. Moreover, the AI‐based system has a significantly larger effect on reducing customer complaints for older and female customers as well as for customers who have had extensive experience using the IVR system. Finally, we find that speech‐recognition failures in customer‐AI interactions lead to increases in customers’ demand for human service and customer complaints. The results from this study provide implications for the implementation of an AI system in call center operations.

Suggested Citation

  • Lingli Wang & Ni Huang & Yili Hong & Luning Liu & Xunhua Guo & Guoqing Chen, 2023. "Voice‐based AI in call center customer service: A natural field experiment," Production and Operations Management, Production and Operations Management Society, vol. 32(4), pages 1002-1018, April.
    • Handle: RePEc:bla:popmgt:v:32:y:2023:i:4:p:1002-1018
    DOI: 10.1111/poms.13953
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/poms.13953
    Download Restriction: no
    File URL: https://libkey.io/10.1111/poms.13953?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Meng Li & Tao Li, 2022. "AI Automation and Retailer Regret in Supply Chains," Production and Operations Management, Production and Operations Management Society, vol. 31(1), pages 83-97, January.
    2. Chiara Longoni & Andrea Bonezzi & Carey K Morewedge, 2019. "Resistance to Medical Artificial Intelligence," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 46(4), pages 629-650.
    3. Tolga Tezcan & Banafsheh Behzad, 2012. "Robust Design and Control of Call Centers with Flexible Interactive Voice Response Systems," Manufacturing & Service Operations Management, INFORMS, vol. 14(3), pages 386-401, July.
    4. Cenying Yang & Yihao Feng & Andrew Whinston, 2022. "Dynamic Pricing and Information Disclosure for Fresh Produce: An Artificial Intelligence Approach," Production and Operations Management, Production and Operations Management Society, vol. 31(1), pages 155-171, January.
    5. Erik Brynjolfsson & Xiang Hui & Meng Liu, 2019. "Does Machine Translation Affect International Trade? Evidence from a Large Digital Platform," Management Science, INFORMS, vol. 65(12), pages 5449-5460, December.
    6. Ruomeng Cui & Santiago Gallino & Antonio Moreno & Dennis J. Zhang, 2018. "The Operational Value of Social Media Information," Production and Operations Management, Production and Operations Management Society, vol. 27(10), pages 1749-1769, October.
    7. Subodha Kumar & Vijay Mookerjee & Abhinav Shubham, 2018. "Research in Operations Management and Information Systems Interface," Production and Operations Management, Production and Operations Management Society, vol. 27(11), pages 1893-1905, November.
    8. Saravanan Kesavan & Bradley R. Staats & Wendell Gilland, 2014. "Volume Flexibility in Services: The Costs and Benefits of Flexible Labor Resources," Management Science, INFORMS, vol. 60(8), pages 1884-1906, August.
    9. List John A., 2007. "Field Experiments: A Bridge between Lab and Naturally Occurring Data," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(2), pages 1-47, April.
    10. Xueming Luo & Siliang Tong & Zheng Fang & Zhe Qu, 2019. "Frontiers: Machines vs. Humans: The Impact of Artificial Intelligence Chatbot Disclosure on Customer Purchases," Marketing Science, INFORMS, vol. 38(6), pages 937-947, November.
    11. Noah Gans & Ger Koole & Avishai Mandelbaum, 2003. "Telephone Call Centers: Tutorial, Review, and Research Prospects," Manufacturing & Service Operations Management, INFORMS, vol. 5(2), pages 79-141, September.
    12. Deborah L. Kellogg & Richard B. Chase, 1995. "Constructing an Empirically Derived Measure for Customer Contact," Management Science, INFORMS, vol. 41(11), pages 1734-1749, November.
    13. Xavier de Groote, 1994. "The Flexibility of Production Processes: A General Framework," Management Science, INFORMS, vol. 40(7), pages 933-945, July.
    14. Paul Resnick & Robert A. Virzi, 1995. "Relief from the Audio Interface Blues: Expanding the Spectrum of Menu, List, and Form Styles," Working Paper Series 184, MIT Center for Coordination Science.
    15. Berkeley J. Dietvorst & Joseph P. Simmons & Cade Massey, 2018. "Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them," Management Science, INFORMS, vol. 64(3), pages 1155-1170, March.
    16. ManMohan S. Sodhi & Zahra Seyedghorban & Hossein Tahernejad & Danny Samson, 2022. "Why emerging supply chain technologies initially disappoint: Blockchain, IoT, and AI," Production and Operations Management, Production and Operations Management Society, vol. 31(6), pages 2517-2537, June.
    17. M. S. Krishnan & Venkatram Ramaswamy & Mary C. Meyer & Paul Damien, 1999. "Customer Satisfaction for Financial Services: The Role of Products, Services, and Information Technology," Management Science, INFORMS, vol. 45(9), pages 1194-1209, September.
    18. Yaşar Levent Koçağa & Mor Armony & Amy R. Ward, 2015. "Staffing Call Centers with Uncertain Arrival Rates and Co-sourcing," Production and Operations Management, Production and Operations Management Society, vol. 24(7), pages 1101-1117, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Siliang Tong & Nan Jia & Xueming Luo & Zheng Fang, 2021. "The Janus face of artificial intelligence feedback: Deployment versus disclosure effects on employee performance," Strategic Management Journal, Wiley Blackwell, vol. 42(9), pages 1600-1631, September.
    2. Zhu, Yimin & Zhang, Jiemin & Wu, Jifei & Liu, Yingyue, 2022. "AI is better when I'm sure: The influence of certainty of needs on consumers' acceptance of AI chatbots," Journal of Business Research, Elsevier, vol. 150(C), pages 642-652.
    3. Peng, Leiqing & Luo, Mengting & Guo, Yulang, 2023. "Deposit AI as the “invisible hand†to make the resale easier: A moderated mediation model," Journal of Retailing and Consumer Services, Elsevier, vol. 75(C).
    4. Chugunova, Marina & Sele, Daniela, 2022. "We and It: An interdisciplinary review of the experimental evidence on how humans interact with machines," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 99(C).
    5. Wang, Cuicui & Li, Yiyang & Fu, Weizhong & Jin, Jia, 2023. "Whether to trust chatbots: Applying the event-related approach to understand consumers’ emotional experiences in interactions with chatbots in e-commerce," Journal of Retailing and Consumer Services, Elsevier, vol. 73(C).
    6. Erik Hermann, 2022. "Leveraging Artificial Intelligence in Marketing for Social Good—An Ethical Perspective," Journal of Business Ethics, Springer, vol. 179(1), pages 43-61, August.
    7. Tinglong Dai & Sridhar Tayur, 2022. "Designing AI‐augmented healthcare delivery systems for physician buy‐in and patient acceptance," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4443-4451, December.
    8. Leah Warfield Smith & Randall Lee Rose & Alex R. Zablah & Heath McCullough & Mohammad “Mike” Saljoughian, 2023. "Examining post-purchase consumer responses to product automation," Journal of the Academy of Marketing Science, Springer, vol. 51(3), pages 530-550, May.
    9. Mahmud, Hasan & Islam, A.K.M. Najmul & Ahmed, Syed Ishtiaque & Smolander, Kari, 2022. "What influences algorithmic decision-making? A systematic literature review on algorithm aversion," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    10. Benedikt Berger & Martin Adam & Alexander Rühr & Alexander Benlian, 2021. "Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(1), pages 55-68, February.
    11. Ekaterina Jussupow & Kai Spohrer & Armin Heinzl & Joshua Gawlitza, 2021. "Augmenting Medical Diagnosis Decisions? An Investigation into Physicians’ Decision-Making Process with Artificial Intelligence," Information Systems Research, INFORMS, vol. 32(3), pages 713-735, September.
    12. repec:cup:judgdm:v:15:y:2020:i:3:p:449-451 is not listed on IDEAS
    13. Scott Schanke & Gordon Burtch & Gautam Ray, 2021. "Estimating the Impact of “Humanizing” Customer Service Chatbots," Information Systems Research, INFORMS, vol. 32(3), pages 736-751, September.
    14. Britt W. J. Mathijsen & A. J. E. M. Janssen & Johan S. H. Leeuwaarden & Bert Zwart, 2018. "Robust heavy-traffic approximations for service systems facing overdispersed demand," Queueing Systems: Theory and Applications, Springer, vol. 90(3), pages 257-289, December.
    15. Zhang, Yaqiong & Wang, Shifu, 2023. "The influence of anthropomorphic appearance of artificial intelligence products on consumer behavior and brand evaluation under different product types," Journal of Retailing and Consumer Services, Elsevier, vol. 74(C).
    16. Chen Yang & Jing Hu, 2022. "When do consumers prefer AI-enabled customer service? The interaction effect of brand personality and service provision type on brand attitudes and purchase intentions," Journal of Brand Management, Palgrave Macmillan, vol. 29(2), pages 167-189, March.
    17. Manjunath Padigar & Ljubomir Pupovac & Ashish Sinha & Rajendra Srivastava, 2022. "The effect of marketing department power on investor responses to announcements of AI-embedded new product innovations," Journal of the Academy of Marketing Science, Springer, vol. 50(6), pages 1277-1298, November.
    18. Suresh P. Sethi & Sushil Gupta & Vipin K. Agrawal & Vijay K. Agrawal, 2022. "Nobel laureates’ contributions to and impacts on operations management," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4283-4303, December.
    19. Alex Mari & Andreina Mandelli & René Algesheimer, 2023. "Shopping with Voice Assistants: How Empathy Affects Individual and Family Decision-Making Outcomes," Working Papers 399, University of Zurich, Department of Business Administration (IBW).
    20. Subodha Kumar & Rakesh R. Mallipeddi, 2022. "Impact of cybersecurity on operations and supply chain management: Emerging trends and future research directions," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4488-4500, December.
    21. Masoud Kamalahmadi & Qiuping Yu & Yong-Pin Zhou, 2021. "Call to Duty: Just-in-Time Scheduling in a Restaurant Chain," Management Science, INFORMS, vol. 67(11), pages 6751-6781, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:popmgt:v:32:y:2023:i:4:p:1002-1018. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1937-5956 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.