IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v67y2019i1p232-249.html
   My bibliography  Save this article

Robust Analysis in Stochastic Simulation: Computation and Performance Guarantees

Author

Listed:
  • Soumyadip Ghosh

    (IBM Research AI, IBM T.J. Watson Research Center, Yorktown Heights, New York 10598)

  • Henry Lam

    (Department of Industrial Engineering and Operations Research, Columbia University, New York, New York 10027)

Abstract

Any performance analysis based on stochastic simulation is subject to the errors inherent in misspecifying the modeling assumptions, particularly the input distributions. In situations with little support from data, we investigate the use of worst-case analysis to analyze these errors, by representing the partial, nonparametric knowledge of the input models via optimization constraints. We study the performance and robustness guarantees of this approach. We design and analyze a numerical scheme for solving a general class of simulation objectives and uncertainty specifications. The key steps involve a randomized discretization of the probability spaces, a simulable unbiased gradient estimator using a nonparametric analog of the likelihood ratio method, and a Frank-Wolfe (FW) variant of the stochastic approximation (SA) method (which we call FWSA) run on the space of input probability distributions. A convergence analysis for FWSA on nonconvex problems is provided. We test the performance of our approach via several numerical examples.

Suggested Citation

  • Soumyadip Ghosh & Henry Lam, 2019. "Robust Analysis in Stochastic Simulation: Computation and Performance Guarantees," Operations Research, INFORMS, vol. 67(1), pages 232-249, January.
    • Handle: RePEc:inm:oropre:v:67:y:2019:i:1:p:232-249
    DOI: 10.1287/opre.2018.1765
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/opre.2018.1765
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.2018.1765?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Uncertainty," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 5, pages 145-154, World Scientific Publishing Co. Pte. Ltd..
    2. James E. Smith, 1995. "Generalized Chebychev Inequalities: Theory and Applications in Decision Analysis," Operations Research, INFORMS, vol. 43(5), pages 807-825, October.
    3. Pengyu Qian & Zizhuo Wang & Zaiwen Wen, 2015. "A Composite Risk Measure Framework for Decision Making under Uncertainty," Papers 1501.01126, arXiv.org.
    4. Rubinstein, Reuven Y., 1986. "The score function approach for sensitivity analysis of computer simulation models," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 28(5), pages 351-379.
    5. Andrew E. B. Lim & J. George Shanthikumar, 2007. "Relative Entropy, Exponential Utility, and Robust Dynamic Pricing," Operations Research, INFORMS, vol. 55(2), pages 198-214, April.
    6. Soumyadip Ghosh & Shane G. Henderson, 2002. "Chessboard Distributions and Random Vectors with Specified Marginals and Covariance Matrix," Operations Research, INFORMS, vol. 50(5), pages 820-834, October.
    7. Philip M. Lurie & Matthew S. Goldberg, 1998. "An Approximate Method for Sampling Correlated Random Variables from Partially-Specified Distributions," Management Science, INFORMS, vol. 44(2), pages 203-218, February.
    8. Paul Glasserman & Linan Yang, 2018. "Bounding Wrong†Way Risk In Cva Calculation," Mathematical Finance, Wiley Blackwell, vol. 28(1), pages 268-305, January.
    9. Stephen E. Chick, 2001. "Input Distribution Selection for Simulation Experiments: Accounting for Input Uncertainty," Operations Research, INFORMS, vol. 49(5), pages 744-758, October.
    10. Joel Goh & Melvyn Sim, 2010. "Distributionally Robust Optimization and Its Tractable Approximations," Operations Research, INFORMS, vol. 58(4-part-1), pages 902-917, August.
    11. Aharon Ben-Tal & Dick den Hertog & Anja De Waegenaere & Bertrand Melenberg & Gijs Rennen, 2013. "Robust Solutions of Optimization Problems Affected by Uncertain Probabilities," Management Science, INFORMS, vol. 59(2), pages 341-357, April.
    12. Arnab Nilim & Laurent El Ghaoui, 2005. "Robust Control of Markov Decision Processes with Uncertain Transition Matrices," Operations Research, INFORMS, vol. 53(5), pages 780-798, October.
    13. Paul Glasserman & Xingbo Xu, 2014. "Robust risk measurement and model risk," Quantitative Finance, Taylor & Francis Journals, vol. 14(1), pages 29-58, January.
    14. John R. Birge & Roger J.-B. Wets, 1987. "Computing Bounds for Stochastic Programming Problems by Means of a Generalized Moment Problem," Mathematics of Operations Research, INFORMS, vol. 12(1), pages 149-162, February.
    15. Paul Glasserman & Xingbo Xu, 2013. "Robust Portfolio Control with Stochastic Factor Dynamics," Operations Research, INFORMS, vol. 61(4), pages 874-893, August.
    16. Henry Lam, 2018. "Sensitivity to Serial Dependency of Input Processes: A Robust Approach," Management Science, INFORMS, vol. 64(3), pages 1311-1327, March.
    17. James E. Smith, 1993. "Moment Methods for Decision Analysis," Management Science, INFORMS, vol. 39(3), pages 340-358, March.
    18. Yuhong Xu, 2014. "Robust valuation and risk measurement under model uncertainty," Papers 1407.8024, arXiv.org.
    19. Erick Delage & Yinyu Ye, 2010. "Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems," Operations Research, INFORMS, vol. 58(3), pages 595-612, June.
    20. Wolfram Wiesemann & Daniel Kuhn & Melvyn Sim, 2014. "Distributionally Robust Convex Optimization," Operations Research, INFORMS, vol. 62(6), pages 1358-1376, December.
    21. Garud N. Iyengar, 2005. "Robust Dynamic Programming," Mathematics of Operations Research, INFORMS, vol. 30(2), pages 257-280, May.
    22. Huan Xu & Shie Mannor, 2012. "Distributionally Robust Markov Decision Processes," Mathematics of Operations Research, INFORMS, vol. 37(2), pages 288-300, May.
    23. Marguerite Frank & Philip Wolfe, 1956. "An algorithm for quadratic programming," Naval Research Logistics Quarterly, John Wiley & Sons, vol. 3(1‐2), pages 95-110, March.
    24. Wei Xie & Barry L. Nelson & Russell R. Barton, 2014. "A Bayesian Framework for Quantifying Uncertainty in Stochastic Simulation," Operations Research, INFORMS, vol. 62(6), pages 1439-1452, December.
    25. Eunhye Song & Barry L. Nelson, 2015. "Quickly Assessing Contributions to Input Uncertainty," IISE Transactions, Taylor & Francis Journals, vol. 47(9), pages 893-909, September.
    26. Martin I. Reiman & Alan Weiss, 1989. "Sensitivity Analysis for Simulations via Likelihood Ratios," Operations Research, INFORMS, vol. 37(5), pages 830-844, October.
    27. Zhaolin Hu & Jing Cao & L. Jeff Hong, 2012. "Robust Simulation of Global Warming Policies Using the DICE Model," Management Science, INFORMS, vol. 58(12), pages 2190-2206, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aleksandrina Goeva & Henry Lam & Huajie Qian & Bo Zhang, 2019. "Optimization-Based Calibration of Simulation Input Models," Operations Research, INFORMS, vol. 67(5), pages 1362-1382, September.
    2. Kim, Sojung & Weber, Stefan, 2022. "Simulation methods for robust risk assessment and the distorted mix approach," European Journal of Operational Research, Elsevier, vol. 298(1), pages 380-398.
    3. Zhaolin Hu & L. Jeff Hong, 2022. "Robust Simulation with Likelihood-Ratio Constrained Input Uncertainty," INFORMS Journal on Computing, INFORMS, vol. 34(4), pages 2350-2367, July.
    4. Sojung Kim & Stefan Weber, 2020. "Simulation Methods for Robust Risk Assessment and the Distorted Mix Approach," Papers 2009.03653, arXiv.org, revised Jan 2022.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aleksandrina Goeva & Henry Lam & Huajie Qian & Bo Zhang, 2019. "Optimization-Based Calibration of Simulation Input Models," Operations Research, INFORMS, vol. 67(5), pages 1362-1382, September.
    2. Jose Blanchet & Karthyek Murthy, 2019. "Quantifying Distributional Model Risk via Optimal Transport," Mathematics of Operations Research, INFORMS, vol. 44(2), pages 565-600, May.
    3. Henry Lam, 2016. "Robust Sensitivity Analysis for Stochastic Systems," Mathematics of Operations Research, INFORMS, vol. 41(4), pages 1248-1275, November.
    4. Henry Lam, 2018. "Sensitivity to Serial Dependency of Input Processes: A Robust Approach," Management Science, INFORMS, vol. 64(3), pages 1311-1327, March.
    5. Zhaolin Hu & L. Jeff Hong, 2022. "Robust Simulation with Likelihood-Ratio Constrained Input Uncertainty," INFORMS Journal on Computing, INFORMS, vol. 34(4), pages 2350-2367, July.
    6. Henry Lam & Clementine Mottet, 2017. "Tail Analysis Without Parametric Models: A Worst-Case Perspective," Operations Research, INFORMS, vol. 65(6), pages 1696-1711, December.
    7. Andrew J. Keith & Darryl K. Ahner, 2021. "A survey of decision making and optimization under uncertainty," Annals of Operations Research, Springer, vol. 300(2), pages 319-353, May.
    8. Corlu, Canan G. & Akcay, Alp & Xie, Wei, 2020. "Stochastic simulation under input uncertainty: A Review," Operations Research Perspectives, Elsevier, vol. 7(C).
    9. Weiwei Fan & L. Jeff Hong & Xiaowei Zhang, 2020. "Distributionally Robust Selection of the Best," Management Science, INFORMS, vol. 66(1), pages 190-208, January.
    10. Postek, Krzysztof & Ben-Tal, A. & den Hertog, Dick & Melenberg, Bertrand, 2015. "Exact Robust Counterparts of Ambiguous Stochastic Constraints Under Mean and Dispersion Information," Other publications TiSEM d718e419-a375-4707-b206-e, Tilburg University, School of Economics and Management.
    11. Cohen, Asaf & Saha, Subhamay, 2021. "Asymptotic optimality of the generalized cμ rule under model uncertainty," Stochastic Processes and their Applications, Elsevier, vol. 136(C), pages 206-236.
    12. Mengshi Lu & Zuo‐Jun Max Shen, 2021. "A Review of Robust Operations Management under Model Uncertainty," Production and Operations Management, Production and Operations Management Society, vol. 30(6), pages 1927-1943, June.
    13. Henry Lam, 2019. "Recovering Best Statistical Guarantees via the Empirical Divergence-Based Distributionally Robust Optimization," Operations Research, INFORMS, vol. 67(4), pages 1090-1105, July.
    14. Postek, Krzysztof & Ben-Tal, A. & den Hertog, Dick & Melenberg, Bertrand, 2015. "Exact Robust Counterparts of Ambiguous Stochastic Constraints Under Mean and Dispersion Information," Discussion Paper 2015-030, Tilburg University, Center for Economic Research.
    15. Zhi Chen & Melvyn Sim & Huan Xu, 2019. "Distributionally Robust Optimization with Infinitely Constrained Ambiguity Sets," Operations Research, INFORMS, vol. 67(5), pages 1328-1344, September.
    16. Maximilian Blesch & Philipp Eisenhauer, 2021. "Robust decision-making under risk and ambiguity," Papers 2104.12573, arXiv.org, revised Oct 2021.
    17. L. Jeff Hong & Zhiyuan Huang & Henry Lam, 2021. "Learning-Based Robust Optimization: Procedures and Statistical Guarantees," Management Science, INFORMS, vol. 67(6), pages 3447-3467, June.
    18. Antonio J. Conejo & Nicholas G. Hall & Daniel Zhuoyu Long & Runhao Zhang, 2021. "Robust Capacity Planning for Project Management," INFORMS Journal on Computing, INFORMS, vol. 33(4), pages 1533-1550, October.
    19. Vishal Gupta, 2019. "Near-Optimal Bayesian Ambiguity Sets for Distributionally Robust Optimization," Management Science, INFORMS, vol. 65(9), pages 4242-4260, September.
    20. Zeynep Turgay & Fikri Karaesmen & Egemen Lerzan Örmeci, 2018. "Structural properties of a class of robust inventory and queueing control problems," Naval Research Logistics (NRL), John Wiley & Sons, vol. 65(8), pages 699-716, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:67:y:2019:i:1:p:232-249. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.