IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i22p4662-d1281461.html
   My bibliography  Save this article

Dynamic Routing Policies for Multi-Skill Call Centers Using Deep Q Network

Author

Listed:
  • Qin Zhang

    (School of Economics and Management, Tongji University, Shanghai 200092, China)

Abstract

When the call center queuing system becomes complex, it turns out that the static routing policy is not optimal. This paper considers the problem of the dynamic routing policy for call centers with multiple skill types and agent groups. A state-dependent routing policy based on the Deep Q Network (DQN) is proposed, and a reinforcement learning algorithm is applied to optimize the routing. A simulation algorithm is designed to help customers and agents interact with the external environment to learn the optimal strategy. The performance evaluation considered in this paper is the service level/abandon rate. Experiments show that the DQN-based dynamic routing policy performs better than the common static policy Global First Come First Serve (FCFS) and the dynamic policy Priorities with Idle Agent Thresholds and Weight-Based Routing in various examples. On the other hand, the training time of the routing policy model based on the DQN is much faster than routing optimization based on simulation and a genetic algorithm.

Suggested Citation

  • Qin Zhang, 2023. "Dynamic Routing Policies for Multi-Skill Call Centers Using Deep Q Network," Mathematics, MDPI, vol. 11(22), pages 1-18, November.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:22:p:4662-:d:1281461
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/22/4662/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/22/4662/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Benjamin Legros, 2018. "Waiting time based routing policies to parallel queues with percentiles objectives," Post-Print hal-02065918, HAL.
    2. John N. Tsitsiklis & Kuang Xu, 2017. "Flexible Queueing Architectures," Operations Research, INFORMS, vol. 65(5), pages 1398-1413, October.
    3. Legros, Benjamin & Jouini, Oualid & Dallery, Yves, 2015. "A flexible architecture for call centers with skill-based routing," International Journal of Production Economics, Elsevier, vol. 159(C), pages 192-207.
    4. Rodney B. Wallace & Ward Whitt, 2005. "A Staffing Algorithm for Call Centers with Skill-Based Routing," Manufacturing & Service Operations Management, INFORMS, vol. 7(4), pages 276-294, August.
    5. Tolga Tezcan & J. G. Dai, 2010. "Dynamic Control of N -Systems with Many Servers: Asymptotic Optimality of a Static Priority Policy in Heavy Traffic," Operations Research, INFORMS, vol. 58(1), pages 94-110, February.
    6. Mehmet Tolga Cezik & Pierre L'Ecuyer, 2008. "Staffing Multiskill Call Centers via Linear Programming and Simulation," Management Science, INFORMS, vol. 54(2), pages 310-323, February.
    7. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Benjamin Legros & Sihan Ding & Rob Mei & Oualid Jouini, 2017. "Call centers with a postponed callback offer," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 39(4), pages 1097-1125, October.
    2. Achal Bassamboo & Assaf Zeevi, 2009. "On a Data-Driven Method for Staffing Large Call Centers," Operations Research, INFORMS, vol. 57(3), pages 714-726, June.
    3. Adan, Ivo J.B.F. & Boon, Marko A.A. & Weiss, Gideon, 2019. "Design heuristic for parallel many server systems," European Journal of Operational Research, Elsevier, vol. 273(1), pages 259-277.
    4. Carri W. Chan & Linda V. Green & Suparerk Lekwijit & Lijian Lu & Gabriel Escobar, 2019. "Assessing the Impact of Service Level When Customer Needs Are Uncertain: An Empirical Investigation of Hospital Step-Down Units," Management Science, INFORMS, vol. 65(2), pages 751-775, February.
    5. Zhen Xu & Hailun Zhang & Jiheng Zhang & Rachel Q. Zhang, 2020. "Online Demand Fulfillment Under Limited Flexibility," Management Science, INFORMS, vol. 66(10), pages 4667-4685, October.
    6. Cong Shi & Yehua Wei & Yuan Zhong, 2019. "Process Flexibility for Multiperiod Production Systems," Operations Research, INFORMS, vol. 67(5), pages 1300-1320, September.
    7. Elvin Coban & Aliza Heching & Alan Scheller‐Wolf, 2019. "Service Center Staffing with Cross‐Trained Agents and Heterogeneous Customers," Production and Operations Management, Production and Operations Management Society, vol. 28(4), pages 788-809, April.
    8. Tom Fangyun Tan & Serguei Netessine, 2019. "When You Work with a Superman, Will You Also Fly? An Empirical Study of the Impact of Coworkers on Performance," Management Science, INFORMS, vol. 65(8), pages 3495-3517, August.
    9. Wyean Chan & Ger Koole & Pierre L'Ecuyer, 2014. "Dynamic Call Center Routing Policies Using Call Waiting and Agent Idle Times," Manufacturing & Service Operations Management, INFORMS, vol. 16(4), pages 544-560, October.
    10. Örmeci, E. Lerzan & Salman, F. Sibel & Yücel, Eda, 2014. "Staff rostering in call centers providing employee transportation," Omega, Elsevier, vol. 43(C), pages 41-53.
    11. Van den Bergh, Jorne & Beliën, Jeroen & De Bruecker, Philippe & Demeulemeester, Erik & De Boeck, Liesje, 2013. "Personnel scheduling: A literature review," European Journal of Operational Research, Elsevier, vol. 226(3), pages 367-385.
    12. Yiting Xing & Ling Li & Zhuming Bi & Marzena Wilamowska‐Korsak & Li Zhang, 2013. "Operations Research (OR) in Service Industries: A Comprehensive Review," Systems Research and Behavioral Science, Wiley Blackwell, vol. 30(3), pages 300-353, May.
    13. Shixin Wang & Xuan Wang & Jiawei Zhang, 2021. "A Review of Flexible Processes and Operations," Production and Operations Management, Production and Operations Management Society, vol. 30(6), pages 1804-1824, June.
    14. Itai Gurvich & James Luedtke & Tolga Tezcan, 2010. "Staffing Call Centers with Uncertain Demand Forecasts: A Chance-Constrained Optimization Approach," Management Science, INFORMS, vol. 56(7), pages 1093-1115, July.
    15. Philipp Afèche & Mojtaba Araghi & Opher Baron, 2017. "Customer Acquisition, Retention, and Service Access Quality: Optimal Advertising, Capacity Level, and Capacity Allocation," Manufacturing & Service Operations Management, INFORMS, vol. 19(4), pages 674-691, October.
    16. Tulika Saha & Sriparna Saha & Pushpak Bhattacharyya, 2020. "Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement learning," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-28, July.
    17. Rouba Ibrahim & Pierre L'Ecuyer, 2013. "Forecasting Call Center Arrivals: Fixed-Effects, Mixed-Effects, and Bivariate Models," Manufacturing & Service Operations Management, INFORMS, vol. 15(1), pages 72-85, May.
    18. Mahmoud Mahfouz & Angelos Filos & Cyrine Chtourou & Joshua Lockhart & Samuel Assefa & Manuela Veloso & Danilo Mandic & Tucker Balch, 2019. "On the Importance of Opponent Modeling in Auction Markets," Papers 1911.12816, arXiv.org.
    19. Achal Bassamboo & J. Michael Harrison & Assaf Zeevi, 2006. "Design and Control of a Large Call Center: Asymptotic Analysis of an LP-Based Method," Operations Research, INFORMS, vol. 54(3), pages 419-435, June.
    20. Imen Azzouz & Wiem Fekih Hassen, 2023. "Optimization of Electric Vehicles Charging Scheduling Based on Deep Reinforcement Learning: A Decentralized Approach," Energies, MDPI, vol. 16(24), pages 1-18, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:22:p:4662-:d:1281461. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.