IDEAS home Printed from
   My bibliography  Save this article

Approximate dynamic programming for capacity allocation in the service industry


  • Schütz, Hans-Jörg
  • Kolisch, Rainer


We consider a problem where different classes of customers can book different types of service in advance and the service company has to respond immediately to the booking request confirming or rejecting it. The objective of the service company is to maximize profit made of class-type specific revenues, refunds for cancellations or no-shows as well as cost of overtime. For the calculation of the latter, information on the underlying appointment schedule is required. In contrast to most models in the literature we assume that the service time of clients is stochastic and that clients might be unpunctual. Throughout the paper we will relate the problem to capacity allocation in radiology services. The problem is modeled as a continuous-time Markov decision process and solved using simulation-based approximate dynamic programming (ADP) combined with a discrete event simulation of the service period. We employ an adapted heuristic ADP algorithm from the literature and investigate on the benefits of applying ADP to this type of problem. First, we study a simplified problem with deterministic service times and punctual arrival of clients and compare the solution from the ADP algorithm to the optimal solution. We find that the heuristic ADP algorithm performs very well in terms of objective function value, solution time, and memory requirements. Second, we study the problem with stochastic service times and unpunctuality. It is then shown that the resulting policy constitutes a large improvement over an “optimal” policy that is deduced using restrictive, simplifying assumptions.

Suggested Citation

  • Schütz, Hans-Jörg & Kolisch, Rainer, 2012. "Approximate dynamic programming for capacity allocation in the service industry," European Journal of Operational Research, Elsevier, vol. 218(1), pages 239-250.
  • Handle: RePEc:eee:ejores:v:218:y:2012:i:1:p:239-250
    DOI: 10.1016/j.ejor.2011.09.007

    Download full text from publisher

    File URL:
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Singh, Sumeetpal S. & Tadic, Vladislav B. & Doucet, Arnaud, 2007. "A policy gradient method for semi-Markov decision processes with application to call admission control," European Journal of Operational Research, Elsevier, vol. 178(3), pages 808-818, May.
    2. Vandaele, Nico & Van Nieuwenhuyse, Inneke & Cupers, Sascha, 2003. "Optimal grouping for a nuclear magnetic resonance scanner by means of an open queueing model," European Journal of Operational Research, Elsevier, vol. 151(1), pages 181-192, November.
    3. Sabine Sickinger & Rainer Kolisch, 2009. "The performance of a generalized Bailey–Welch rule for outpatient appointment scheduling under inpatient and emergency demand," Health Care Management Science, Springer, vol. 12(4), pages 408-419, December.
    4. Gosavi, Abhijit, 2004. "Reinforcement learning for long-run average cost," European Journal of Operational Research, Elsevier, vol. 155(3), pages 654-674, June.
    5. Tapas K. Das & Abhijit Gosavi & Sridhar Mahadevan & Nicholas Marchalleck, 1999. "Solving Semi-Markov Decision Problems Using Average Reward Reinforcement Learning," Management Science, INFORMS, vol. 45(4), pages 560-574, April.
    6. Yigal Gerchak & Diwakar Gupta & Mordechai Henig, 1996. "Reservation Planning for Elective Surgery Under Uncertain Demand for Emergency Surgery," Management Science, INFORMS, vol. 42(3), pages 321-334, March.
    7. Nan Liu & Serhan Ziya & Vidyadhar G. Kulkarni, 2010. "Dynamic Scheduling of Outpatient Appointments Under Patient No-Shows and Cancellations," Manufacturing & Service Operations Management, INFORMS, vol. 12(2), pages 347-364, September.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Qing, Qiankai & Deng, Tianhu & Wang, Hongwei, 2017. "Capacity allocation under downstream competition and bargaining," European Journal of Operational Research, Elsevier, vol. 261(1), pages 97-107.
    2. Gartner, Daniel & Kolisch, Rainer, 2014. "Scheduling the hospital-wide flow of elective patients," European Journal of Operational Research, Elsevier, vol. 233(3), pages 689-699.
    3. De Vuyst, Stijn & Bruneel, Herwig & Fiems, Dieter, 2014. "Computationally efficient evaluation of appointment schedules in health care," European Journal of Operational Research, Elsevier, vol. 237(3), pages 1142-1154.
    4. Samorani, Michele & LaGanga, Linda R., 2015. "Outpatient appointment scheduling given individual day-dependent no-show predictions," European Journal of Operational Research, Elsevier, vol. 240(1), pages 245-257.
    5. Qu, Xiuli & Peng, Yidong & Shi, Jing & LaGanga, Linda, 2015. "An MDP model for walk-in patient admission management in primary care clinics," International Journal of Production Economics, Elsevier, vol. 168(C), pages 303-320.
    6. Astaraky, Davood & Patrick, Jonathan, 2015. "A simulation based approximate dynamic programming approach to multi-class, multi-resource surgical scheduling," European Journal of Operational Research, Elsevier, vol. 245(1), pages 309-319.
    7. Sauré, Antoine & Patrick, Jonathan & Tyldesley, Scott & Puterman, Martin L., 2012. "Dynamic multi-appointment patient scheduling for radiation therapy," European Journal of Operational Research, Elsevier, vol. 223(2), pages 573-584.
    8. repec:kap:hcarem:v:20:y:2017:i:2:d:10.1007_s10729-016-9354-6 is not listed on IDEAS
    9. Ahmadi-Javid, Amir & Jalali, Zahra & Klassen, Kenneth J, 2017. "Outpatient appointment systems in healthcare: A review of optimization studies," European Journal of Operational Research, Elsevier, vol. 258(1), pages 3-34.
    10. Yin, Jiateng & Tang, Tao & Yang, Lixing & Gao, Ziyou & Ran, Bin, 2016. "Energy-efficient metro train rescheduling with uncertain time-variant passenger demands: An approximate dynamic programming approach," Transportation Research Part B: Methodological, Elsevier, vol. 91(C), pages 178-210.
    11. Geng, Na & Xie, Xiaolan & Jiang, Zhibin, 2013. "Implementation strategies of a contract-based MRI examination reservation process for stroke patients," European Journal of Operational Research, Elsevier, vol. 231(2), pages 371-380.


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:218:y:2012:i:1:p:239-250. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.