IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v218y2012i1p239-250.html
   My bibliography  Save this article

Approximate dynamic programming for capacity allocation in the service industry

Author

Listed:
  • Schütz, Hans-Jörg
  • Kolisch, Rainer

Abstract

We consider a problem where different classes of customers can book different types of service in advance and the service company has to respond immediately to the booking request confirming or rejecting it. The objective of the service company is to maximize profit made of class-type specific revenues, refunds for cancellations or no-shows as well as cost of overtime. For the calculation of the latter, information on the underlying appointment schedule is required. In contrast to most models in the literature we assume that the service time of clients is stochastic and that clients might be unpunctual. Throughout the paper we will relate the problem to capacity allocation in radiology services. The problem is modeled as a continuous-time Markov decision process and solved using simulation-based approximate dynamic programming (ADP) combined with a discrete event simulation of the service period. We employ an adapted heuristic ADP algorithm from the literature and investigate on the benefits of applying ADP to this type of problem. First, we study a simplified problem with deterministic service times and punctual arrival of clients and compare the solution from the ADP algorithm to the optimal solution. We find that the heuristic ADP algorithm performs very well in terms of objective function value, solution time, and memory requirements. Second, we study the problem with stochastic service times and unpunctuality. It is then shown that the resulting policy constitutes a large improvement over an “optimal” policy that is deduced using restrictive, simplifying assumptions.

Suggested Citation

  • Schütz, Hans-Jörg & Kolisch, Rainer, 2012. "Approximate dynamic programming for capacity allocation in the service industry," European Journal of Operational Research, Elsevier, vol. 218(1), pages 239-250.
    • Handle: RePEc:eee:ejores:v:218:y:2012:i:1:p:239-250
    DOI: 10.1016/j.ejor.2011.09.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221711008101
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2011.09.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Janakiram Subramanian & Shaler Stidham & Conrad J. Lautenbacher, 1999. "Airline Yield Management with Overbooking, Cancellations, and No-Shows," Transportation Science, INFORMS, vol. 33(2), pages 147-167, May.
    2. Daniela Pucci de Farias & Benjamin Van Roy, 2006. "A Cost-Shaping Linear Program for Average-Cost Approximate Dynamic Programming with Performance Guarantees," Mathematics of Operations Research, INFORMS, vol. 31(3), pages 597-620, August.
    3. Tak C. Lee & Marvin Hersh, 1993. "A Model for Dynamic Airline Seat Inventory Control with Multiple Seat Bookings," Transportation Science, INFORMS, vol. 27(3), pages 252-265, August.
    4. Singh, Sumeetpal S. & Tadic, Vladislav B. & Doucet, Arnaud, 2007. "A policy gradient method for semi-Markov decision processes with application to call admission control," European Journal of Operational Research, Elsevier, vol. 178(3), pages 808-818, May.
    5. Santanu Chakraborty & Kumar Muthuraman & Mark Lawley, 2010. "Sequential clinical scheduling with patient no-shows and general service time distributions," IISE Transactions, Taylor & Francis Journals, vol. 42(5), pages 354-366.
    6. Sabine Sickinger & Rainer Kolisch, 2009. "The performance of a generalized Bailey–Welch rule for outpatient appointment scheduling under inpatient and emergency demand," Health Care Management Science, Springer, vol. 12(4), pages 408-419, December.
    7. Daniela Pucci de Farias & Benjamin Van Roy, 2004. "On Constraint Sampling in the Linear Programming Approach to Approximate Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 29(3), pages 462-478, August.
    8. Gosavi, Abhijit, 2004. "Reinforcement learning for long-run average cost," European Journal of Operational Research, Elsevier, vol. 155(3), pages 654-674, June.
    9. Yigal Gerchak & Diwakar Gupta & Mordechai Henig, 1996. "Reservation Planning for Elective Surgery Under Uncertain Demand for Emergency Surgery," Management Science, INFORMS, vol. 42(3), pages 321-334, March.
    10. Daniel Adelman, 2007. "Dynamic Bid Prices in Revenue Management," Operations Research, INFORMS, vol. 55(4), pages 647-661, August.
    11. Bo Zeng & Ayten Turkcan & Ji Lin & Mark Lawley, 2010. "Clinic scheduling models with overbooking for patients with heterogeneous no-show probabilities," Annals of Operations Research, Springer, vol. 178(1), pages 121-144, July.
    12. D. P. de Farias & B. Van Roy, 2003. "The Linear Programming Approach to Approximate Dynamic Programming," Operations Research, INFORMS, vol. 51(6), pages 850-865, December.
    13. Vandaele, Nico & Van Nieuwenhuyse, Inneke & Cupers, Sascha, 2003. "Optimal grouping for a nuclear magnetic resonance scanner by means of an open queueing model," European Journal of Operational Research, Elsevier, vol. 151(1), pages 181-192, November.
    14. Linda V. Green & Sergei Savin & Ben Wang, 2006. "Managing Patient Service in a Diagnostic Medical Facility," Operations Research, INFORMS, vol. 54(1), pages 11-25, February.
    15. Diwakar Gupta & Lei Wang, 2008. "Revenue Management for a Primary-Care Clinic in the Presence of Patient Choice," Operations Research, INFORMS, vol. 56(3), pages 576-592, June.
    16. Alexander Erdelyi & Huseyin Topaloglu, 2009. "Computing protection level policies for dynamic capacity allocation problems by using stochastic approximation methods," IISE Transactions, Taylor & Francis Journals, vol. 41(6), pages 498-510.
    17. Tapas K. Das & Abhijit Gosavi & Sridhar Mahadevan & Nicholas Marchalleck, 1999. "Solving Semi-Markov Decision Problems Using Average Reward Reinforcement Learning," Management Science, INFORMS, vol. 45(4), pages 560-574, April.
    18. Nan Liu & Serhan Ziya & Vidyadhar G. Kulkarni, 2010. "Dynamic Scheduling of Outpatient Appointments Under Patient No-Shows and Cancellations," Manufacturing & Service Operations Management, INFORMS, vol. 12(2), pages 347-364, September.
    19. VANDAELE, Nico & AN NIEUWENHUYSE, Inneke & CUPERS, Sascha, 2003. "Optimal grouping for a nuclear magnetic resonance scanner by means of an open queueing model," LIDAM Reprints CORE 1813, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gartner, Daniel & Kolisch, Rainer, 2014. "Scheduling the hospital-wide flow of elective patients," European Journal of Operational Research, Elsevier, vol. 233(3), pages 689-699.
    2. Samorani, Michele & LaGanga, Linda R., 2015. "Outpatient appointment scheduling given individual day-dependent no-show predictions," European Journal of Operational Research, Elsevier, vol. 240(1), pages 245-257.
    3. Peter J. H. Hulshof & Martijn R. K. Mes & Richard J. Boucherie & Erwin W. Hans, 2016. "Patient admission planning using Approximate Dynamic Programming," Flexible Services and Manufacturing Journal, Springer, vol. 28(1), pages 30-61, June.
    4. Astaraky, Davood & Patrick, Jonathan, 2015. "A simulation based approximate dynamic programming approach to multi-class, multi-resource surgical scheduling," European Journal of Operational Research, Elsevier, vol. 245(1), pages 309-319.
    5. Alireza F. Hesaraki & Nico P. Dellaert & Ton Kok, 2020. "Integrating nurse assignment in outpatient chemotherapy appointment scheduling," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 42(4), pages 935-963, December.
    6. Shuwan Zhu & Wenjuan Fan & Shanlin Yang & Jun Pei & Panos M. Pardalos, 2019. "Operating room planning and surgical case scheduling: a review of literature," Journal of Combinatorial Optimization, Springer, vol. 37(3), pages 757-805, April.
    7. Yin, Jiateng & Tang, Tao & Yang, Lixing & Gao, Ziyou & Ran, Bin, 2016. "Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach," Transportation Research Part B: Methodological, Elsevier, vol. 91(C), pages 178-210.
    8. Ulusan, Aybike & Ergun, Özlem, 2021. "Approximate dynamic programming for network recovery problems with stochastic demand," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 151(C).
    9. Geng, Na & Xie, Xiaolan & Jiang, Zhibin, 2013. "Implementation strategies of a contract-based MRI examination reservation process for stroke patients," European Journal of Operational Research, Elsevier, vol. 231(2), pages 371-380.
    10. Qing, Qiankai & Deng, Tianhu & Wang, Hongwei, 2017. "Capacity allocation under downstream competition and bargaining," European Journal of Operational Research, Elsevier, vol. 261(1), pages 97-107.
    11. De Vuyst, Stijn & Bruneel, Herwig & Fiems, Dieter, 2014. "Computationally efficient evaluation of appointment schedules in health care," European Journal of Operational Research, Elsevier, vol. 237(3), pages 1142-1154.
    12. Qu, Xiuli & Peng, Yidong & Shi, Jing & LaGanga, Linda, 2015. "An MDP model for walk-in patient admission management in primary care clinics," International Journal of Production Economics, Elsevier, vol. 168(C), pages 303-320.
    13. Sauré, Antoine & Patrick, Jonathan & Tyldesley, Scott & Puterman, Martin L., 2012. "Dynamic multi-appointment patient scheduling for radiation therapy," European Journal of Operational Research, Elsevier, vol. 223(2), pages 573-584.
    14. Hans-Jörg Schütz & Rainer Kolisch, 2013. "Capacity allocation for demand of different customer-product-combinations with cancellations, no-shows, and overbooking when there is a sequential delivery of service," Annals of Operations Research, Springer, vol. 206(1), pages 401-423, July.
    15. Abdolreza Rasouli Kenari & Mahboubeh Shamsi, 2021. "A hyper-heuristic selector algorithm for cloud computing scheduling based on workflow features," OPSEARCH, Springer;Operational Research Society of India, vol. 58(4), pages 852-868, December.
    16. Ridvan Gedik & Shengfan Zhang & Chase Rainwater, 2017. "Strategic level proton therapy patient admission planning: a Markov decision process modeling approach," Health Care Management Science, Springer, vol. 20(2), pages 286-302, June.
    17. Ahmadi-Javid, Amir & Jalali, Zahra & Klassen, Kenneth J, 2017. "Outpatient appointment systems in healthcare: A review of optimization studies," European Journal of Operational Research, Elsevier, vol. 258(1), pages 3-34.
    18. Tu San Pham & Antoine Legrain & Patrick De Causmaecker & Louis-Martin Rousseau, 2023. "A Prediction-Based Approach for Online Dynamic Appointment Scheduling: A Case Study in Radiotherapy Treatment," INFORMS Journal on Computing, INFORMS, vol. 35(4), pages 844-868, July.
    19. Huyang Xu & Yuanchen Fang & Chun-An Chou & Nasser Fard & Li Luo, 2023. "A reinforcement learning-based optimal control approach for managing an elective surgery backlog after pandemic disruption," Health Care Management Science, Springer, vol. 26(3), pages 430-446, September.
    20. Antoine Sauré & Jonathan Patrick & Martin L. Puterman, 2015. "Simulation-Based Approximate Policy Iteration with Generalized Logistic Functions," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 579-595, August.
    21. Creemers, Stefan & Lambrecht, Marc R. & Beliën, Jeroen & Van den Broeke, Maud, 2021. "Evaluation of appointment scheduling rules: A multi-performance measurement approach," Omega, Elsevier, vol. 100(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hans-Jörg Schütz & Rainer Kolisch, 2013. "Capacity allocation for demand of different customer-product-combinations with cancellations, no-shows, and overbooking when there is a sequential delivery of service," Annals of Operations Research, Springer, vol. 206(1), pages 401-423, July.
    2. Ahmadi-Javid, Amir & Jalali, Zahra & Klassen, Kenneth J, 2017. "Outpatient appointment systems in healthcare: A review of optimization studies," European Journal of Operational Research, Elsevier, vol. 258(1), pages 3-34.
    3. Van-Anh Truong, 2015. "Optimal Advance Scheduling," Management Science, INFORMS, vol. 61(7), pages 1584-1597, July.
    4. Yongbo Xiao & Yan Zhu, 2016. "Value management of diagnostic equipment with cancelation, no‐show, and emergency patients," Naval Research Logistics (NRL), John Wiley & Sons, vol. 63(4), pages 287-304, June.
    5. Paola Cappanera & Filippo Visintin & Carlo Banditori & Daniele Feo, 2019. "Evaluating the long-term effects of appointment scheduling policies in a magnetic resonance imaging setting," Flexible Services and Manufacturing Journal, Springer, vol. 31(1), pages 212-254, March.
    6. Qu, Xiuli & Peng, Yidong & Shi, Jing & LaGanga, Linda, 2015. "An MDP model for walk-in patient admission management in primary care clinics," International Journal of Production Economics, Elsevier, vol. 168(C), pages 303-320.
    7. Jianzhe Luo & Vidyadhar G. Kulkarni & Serhan Ziya, 2012. "Appointment Scheduling Under Patient No-Shows and Service Interruptions," Manufacturing & Service Operations Management, INFORMS, vol. 14(4), pages 670-684, October.
    8. Zhuang, Weifen & Li, Michael Z.F., 2012. "Monotone optimal control for a class of Markov decision processes," European Journal of Operational Research, Elsevier, vol. 217(2), pages 342-350.
    9. Miao Bai & Bjorn Berg & Esra Sisikoglu Sir & Mustafa Y. Sir, 2023. "Partially partitioned templating strategies for outpatient specialty practices," Production and Operations Management, Production and Operations Management Society, vol. 32(1), pages 301-318, January.
    10. Nan Liu & Peter M. van de Ven & Bo Zhang, 2019. "Managing Appointment Booking Under Customer Choices," Management Science, INFORMS, vol. 65(9), pages 4280-4298, September.
    11. Thomas W. M. Vossen & Dan Zhang, 2015. "Reductions of Approximate Linear Programs for Network Revenue Management," Operations Research, INFORMS, vol. 63(6), pages 1352-1371, December.
    12. Sauré, Antoine & Patrick, Jonathan & Tyldesley, Scott & Puterman, Martin L., 2012. "Dynamic multi-appointment patient scheduling for radiation therapy," European Journal of Operational Research, Elsevier, vol. 223(2), pages 573-584.
    13. Jonathan Patrick & Martin L. Puterman & Maurice Queyranne, 2008. "Dynamic Multipriority Patient Scheduling for a Diagnostic Resource," Operations Research, INFORMS, vol. 56(6), pages 1507-1525, December.
    14. Meissner, Joern & Strauss, Arne, 2012. "Network revenue management with inventory-sensitive bid prices and customer choice," European Journal of Operational Research, Elsevier, vol. 216(2), pages 459-468.
    15. Katsumi Morikawa & Katsuhiko Takahashi & Daisuke Hirotani, 2018. "Performance evaluation of candidate appointment schedules using clearing functions," Journal of Intelligent Manufacturing, Springer, vol. 29(3), pages 509-518, March.
    16. Michael H. Veatch, 2013. "Approximate Linear Programming for Average Cost MDPs," Mathematics of Operations Research, INFORMS, vol. 38(3), pages 535-544, August.
    17. Dimitris Bertsimas & Velibor V. Mišić, 2016. "Decomposable Markov Decision Processes: A Fluid Optimization Approach," Operations Research, INFORMS, vol. 64(6), pages 1537-1555, December.
    18. Jiang, Yangzi & Abouee-Mehrizi, Hossein & Diao, Yuhe, 2020. "Data-driven analytics to support scheduling of multi-priority multi-class patients with wait time targets," European Journal of Operational Research, Elsevier, vol. 281(3), pages 597-611.
    19. Dongyang Wang & Kumar Muthuraman & Douglas Morrice, 2019. "Coordinated Patient Appointment Scheduling for a Multistation Healthcare Network," Operations Research, INFORMS, vol. 67(3), pages 599-618, May.
    20. Christos Zacharias & Tallys Yunes, 2020. "Multimodularity in the Stochastic Appointment Scheduling Problem with Discrete Arrival Epochs," Management Science, INFORMS, vol. 66(2), pages 744-763, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:218:y:2012:i:1:p:239-250. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.