IDEAS home Printed from https://ideas.repec.org/a/wly/navres/v56y2009i3p239-249.html
   My bibliography  Save this article

What you should know about approximate dynamic programming

Author

Listed:
  • Warren B. Powell

Abstract

Approximate dynamic programming (ADP) is a broad umbrella for a modeling and algorithmic strategy for solving problems that are sometimes large and complex, and are usually (but not always) stochastic. It is most often presented as a method for overcoming the classic curse of dimensionality that is well‐known to plague the use of Bellman's equation. For many problems, there are actually up to three curses of dimensionality. But the richer message of approximate dynamic programming is learning what to learn, and how to learn it, to make better decisions over time. This article provides a brief review of approximate dynamic programming, without intending to be a complete tutorial. Instead, our goal is to provide a broader perspective of ADP and how it should be approached from the perspective of different problem classes. © 2009 Wiley Periodicals, Inc. Naval Research Logistics 2009

Suggested Citation

  • Warren B. Powell, 2009. "What you should know about approximate dynamic programming," Naval Research Logistics (NRL), John Wiley & Sons, vol. 56(3), pages 239-249, April.
    • Handle: RePEc:wly:navres:v:56:y:2009:i:3:p:239-249
    DOI: 10.1002/nav.20347
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/nav.20347
    Download Restriction: no
    File URL: https://libkey.io/10.1002/nav.20347?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Michael C. Fu, 2002. "Feature Article: Optimization for simulation: Theory vs. Practice," INFORMS Journal on Computing, INFORMS, vol. 14(3), pages 192-215, August.
    2. Kenneth L. Judd, 1998. "Numerical Methods in Economics," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262100711, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rempel, M. & Cai, J., 2021. "A review of approximate dynamic programming applications within military operations research," Operations Research Perspectives, Elsevier, vol. 8(C).
    2. Xiao, Baichun & Yang, Wei, 2021. "A Bayesian learning model for estimating unknown demand parameter in revenue management," European Journal of Operational Research, Elsevier, vol. 293(1), pages 248-262.
    3. Marynissen, Joren & Demeulemeester, Erik, 2019. "Literature review on multi-appointment scheduling problems in hospitals," European Journal of Operational Research, Elsevier, vol. 272(2), pages 407-419.
    4. Rossi, Roberto & Tomasella, Maurizio & Martin-Barragan, Belen & Embley, Tim & Walsh, Christopher & Langston, Matthew, 2019. "The Dynamic Bowser Routing Problem," European Journal of Operational Research, Elsevier, vol. 275(1), pages 108-126.
    5. Carbonneau, Alexandre, 2021. "Deep hedging of long-term financial derivatives," Insurance: Mathematics and Economics, Elsevier, vol. 99(C), pages 327-340.
    6. Alexandre Carbonneau, 2020. "Deep Hedging of Long-Term Financial Derivatives," Papers 2007.15128, arXiv.org.
    7. Lauer, Christopher J. & Montgomery, Claire A. & Dietterich, Thomas G., 2017. "Spatial interactions and optimal forest management on a fire-threatened landscape," Forest Policy and Economics, Elsevier, vol. 83(C), pages 107-120.
    8. Alexandre Carbonneau & Fr'ed'eric Godin, 2021. "Deep Equal Risk Pricing of Financial Derivatives with Multiple Hedging Instruments," Papers 2102.12694, arXiv.org.
    9. Cervellera, Cristiano, 2023. "Optimized ensemble value function approximation for dynamic programming," European Journal of Operational Research, Elsevier, vol. 309(2), pages 719-730.
    10. Alexandra M. Newman & Martin Weiss, 2013. "A Survey of Linear and Mixed-Integer Optimization Tutorials," INFORMS Transactions on Education, INFORMS, vol. 14(1), pages 26-38, September.
    11. Daniel Egan & Qilun Zhu & Robert Prucka, 2023. "A Review of Reinforcement Learning-Based Powertrain Controllers: Effects of Agent Selection for Mixed-Continuity Control and Reward Formulation," Energies, MDPI, vol. 16(8), pages 1-31, April.
    12. Nikola Mardešić & Tomislav Erdelić & Tonči Carić & Marko Đurasević, 2023. "Review of Stochastic Dynamic Vehicle Routing in the Evolving Urban Logistics Environment," Mathematics, MDPI, vol. 12(1), pages 1-44, December.
    13. Gökalp, E. & Gülpınar, N. & Doan, X.V., 2023. "Dynamic surgery management under uncertainty," European Journal of Operational Research, Elsevier, vol. 309(2), pages 832-844.
    14. Mojtaba Heydar & Małgorzata M. O’Reilly & Erin Trainer & Mark Fackrell & Peter G. Taylor & Ali Tirdad, 2022. "A stochastic model for the patient-bed assignment problem with random arrivals and departures," Annals of Operations Research, Springer, vol. 315(2), pages 813-845, August.
    15. Christopher Dance & Alexei Gaivoronski, 2012. "Stochastic optimization for real time service capacity allocation under random service demand," Annals of Operations Research, Springer, vol. 193(1), pages 221-253, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    2. Richard Pierse, 2006. "Optimal control in nonlinear models: a generalised Gauss-Newton algorithm with analytic derivatives," School of Economics Discussion Papers 0906, School of Economics, University of Surrey.
    3. Andrew Patton, 2002. "(IAM Series No 001) On the Out-Of-Sample Importance of Skewness and Asymetric Dependence for Asset Allocation," FMG Discussion Papers dp431, Financial Markets Group.
    4. Francisco Gallego & Andrés Hernando, 2009. "School Choice in Chile: Looking at the Demand Side," Documentos de Trabajo 356, Instituto de Economia. Pontificia Universidad Católica de Chile..
    5. Noordhoek, Marije & Dullaert, Wout & Lai, David S.W. & de Leeuw, Sander, 2018. "A simulation–optimization approach for a service-constrained multi-echelon distribution network," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 114(C), pages 292-311.
    6. Claude Hillinger, 2002. "A General Theory of Price and Quantity Aggregation and Welfare Measurement," CESifo Working Paper Series 818, CESifo.
    7. Salerno, Gillian & Beard, Rodney & McDonald, Stuart, 2007. "Rent Seeking Behavior and Optimal Taxation of Pollution in Shallow Lakes," MPRA Paper 11225, University Library of Munich, Germany, revised 22 Oct 2008.
    8. Maria Casanova-Rivas, 2008. "Dynamic Complementarities: A Computational and Empirical Analysis of Couples' Retirement Decisions," 2008 Meeting Papers 1073, Society for Economic Dynamics.
    9. Heer, Burkhard & Polito, Vito & Wickens, Michael R., 2020. "Population aging, social security and fiscal limits," Journal of Economic Dynamics and Control, Elsevier, vol. 116(C).
    10. Andreas Lanz & Gregor Reich & Ole Wilms, 2022. "Adaptive grids for the estimation of dynamic models," Quantitative Marketing and Economics (QME), Springer, vol. 20(2), pages 179-238, June.
    11. Jacques Le Cacheux & Vincent Touzé, 2002. "Les modèles d'équilibre général calculable à générations imbriquées. Enjeux, méthodes et résultats," Revue de l'OFCE, Presses de Sciences-Po, vol. 80(1), pages 87-113.
    12. Karantounias, Anastasios G., 2023. "Doubts about the model and optimal policy," Journal of Economic Theory, Elsevier, vol. 210(C).
    13. Pelin Ilbas, 2006. "Optimal Monetary Policy rules for the Euro area in a DSGE framework," Working Papers of Department of Economics, Leuven ces0613, KU Leuven, Faculty of Economics and Business (FEB), Department of Economics, Leuven.
    14. Atanas Christev, 2006. "Learning Hyperinflations," Computing in Economics and Finance 2006 475, Society for Computational Economics.
    15. Kollmann, Robert, 2003. "Monetary Policy Rules in an Interdependent World," CEPR Discussion Papers 4012, C.E.P.R. Discussion Papers.
    16. Zheng, Liang & Xue, Xinfeng & Xu, Chengcheng & Ran, Bin, 2019. "A stochastic simulation-based optimization method for equitable and efficient network-wide signal timing under uncertainties," Transportation Research Part B: Methodological, Elsevier, vol. 122(C), pages 287-308.
    17. Borovička, Jaroslav & Hansen, Lars Peter, 2014. "Examining macroeconomic models through the lens of asset pricing," Journal of Econometrics, Elsevier, vol. 183(1), pages 67-90.
    18. Frölich, Markus & Lechner, Michael, 2010. "Exploiting Regional Treatment Intensity for the Evaluation of Labor Market Policies," Journal of the American Statistical Association, American Statistical Association, vol. 105(491), pages 1014-1029.
    19. Nikolaj Malchow-Møller & Michael Svarer, 2003. "Estimation of the multinomial logit model with random effects," Applied Economics Letters, Taylor & Francis Journals, vol. 10(7), pages 389-392.
    20. Röhrs, Sigrid & Winter, Christoph, 2017. "Reducing government debt in the presence of inequality," Journal of Economic Dynamics and Control, Elsevier, vol. 82(C), pages 1-20.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:navres:v:56:y:2009:i:3:p:239-249. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)1520-6750 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.