IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v31y2019i4p745-770.html
   My bibliography  Save this article

An Efficient Morris Method-Based Framework for Simulation Factor Screening

Author

Listed:
  • Wen Shi

    (School of Business, Central South University, Changsha, Hunan 410083, China;)

  • Xi Chen

    (Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, Virginia 24061;)

  • Jennifer Shang

    (Business and College of Business Administration, Katz Graduate School of Business, University of Pittsburgh, Pittsburgh, Pennsylvania 15260)

Abstract

In this paper, we study the methodological underpinnings of the Morris elementary effects method, a model-free factor-screening technique originally proposed for deterministic simulation experiments, and develop an efficient Morris method–based framework (EMM) for simulation factor screening. Equipped with an efficient cluster-sampling procedure, EMM can simultaneously screen the main and interaction (or nonlinear) effects of all factors and control the overall false discovery rate at a prescribed level. Despite focusing on deterministic simulation experiments, we reveal the connections between EMM (also the Morris method) and other factor-screening methods, such as sequential bifurcation, and examine the resulting implications in the stochastic simulation setting under some commonly stipulated assumptions in design of experiments. Numerical experiments are presented to demonstrate the efficiency and efficacy of EMM.

Suggested Citation

  • Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    • Handle: RePEc:inm:orijoc:v:31:y:2019:i:4:p:745-770
    DOI: 10.1287/ijoc.2018.0836
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/ijoc.2018.0836
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2018.0836?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Marc Kennedy & Clive Anderson & Anthony O'Hagan & Mark Lomas & Ian Woodward & John Paul Gosling & Andreas Heinemeyer, 2008. "Quantifying uncertainty in the biospheric carbon flux for England and Wales," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(1), pages 109-135, January.
    2. Michail Tsagris & Christina Beneki & Hossein Hassani, 2014. "On the Folded Normal Distribution," Mathematics, MDPI, vol. 2(1), pages 1-17, February.
    3. Shi, Wen & Kleijnen, Jack P.C. & Liu, Zhixue, 2014. "Factor screening for simulation with multiple responses: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 237(1), pages 136-147.
    4. James R. Carpenter & Harvey Goldstein & Jon Rasbash, 2003. "A novel bootstrap procedure for assessing the relationship between class size and achievement," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 52(4), pages 431-443, October.
    5. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2010. "Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 482-492, August.
    6. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    7. Russell Davidson & James MacKinnon, 2000. "Bootstrap tests: how many bootstraps?," Econometric Reviews, Taylor & Francis Journals, vol. 19(1), pages 55-68.
    8. Kleijnen, Jack P.C. & Pierreval, Henri & Zhang, Jin, 2011. "Methodology for determining the acceptability of system designs in uncertain environments," European Journal of Operational Research, Elsevier, vol. 209(2), pages 176-183, March.
    9. Mark Broadie & Yiping Du & Ciamac C. Moallemi, 2011. "Efficient Risk Estimation via Nested Sequential Simulation," Management Science, INFORMS, vol. 57(6), pages 1172-1194, June.
    10. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    11. Morris, Max D., 2006. "Input screening: Finding the important model inputs on a budget," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1252-1256.
    12. Hu, James X. & Zhao, Hongyu & Zhou, Harrison H., 2010. "False Discovery Rate Control With Groups," Journal of the American Statistical Association, American Statistical Association, vol. 105(491), pages 1215-1227.
    13. Pujol, Gilles, 2009. "Simplex-based screening designs for estimating metamodels," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1156-1160.
    14. Jack P.C. Kleijnen, 2015. "Design and Analysis of Simulation Experiments," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-18087-8, September.
    15. Russell R. Barton & Barry L. Nelson & Wei Xie, 2014. "Quantifying Input Uncertainty via Simulation Confidence Intervals," INFORMS Journal on Computing, INFORMS, vol. 26(1), pages 74-87, February.
    16. Efron B. & Tibshirani R. & Storey J.D. & Tusher V., 2001. "Empirical Bayes Analysis of a Microarray Experiment," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1151-1160, December.
    17. C. A. Field & A. H. Welsh, 2007. "Bootstrapping clustered data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(3), pages 369-390, June.
    18. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2006. "Controlled Sequential Bifurcation: A New Factor-Screening Method for Discrete-Event Simulation," Operations Research, INFORMS, vol. 54(4), pages 743-755, August.
    19. Sanchez, Susan M. & Moeeni, Farhad & Sanchez, Paul J., 2006. "So many factors, so little time...Simulation experiments in the frequency domain," International Journal of Production Economics, Elsevier, vol. 103(1), pages 149-165, September.
    20. Shiquan Ren & Hong Lai & Wenjing Tong & Mostafa Aminzadeh & Xuezhang Hou & Shenghan Lai, 2010. "Nonparametric bootstrapping for hierarchical data," Journal of Applied Statistics, Taylor & Francis Journals, vol. 37(9), pages 1487-1498.
    21. John D. Storey, 2002. "A direct approach to false discovery rates," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(3), pages 479-498, August.
    22. Shen, Hua & Wan, Hong, 2009. "Controlled sequential factorial design for simulation factor screening," European Journal of Operational Research, Elsevier, vol. 198(2), pages 511-519, October.
    23. Jin, Jiashun & Cai, T. Tony, 2007. "Estimating the Null and the Proportion of Nonnull Effects in Large-Scale Multiple Comparisons," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 495-506, June.
    24. Efron, Bradley, 2007. "Correlation and Large-Scale Simultaneous Significance Testing," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 93-103, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shi, Wen & Zhou, Qing & Zhou, Yanju, 2023. "An efficient elementary effect-based method for sensitivity analysis in identifying main and two-factor interaction effects," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    2. Shi, Wen & Chen, Xi, 2019. "Controlled Morris method: A new factor screening approach empowered by a distribution-free sequential multiple testing procedure," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 299-314.
    3. Borgonovo, Emanuele & Rabitti, Giovanni, 2023. "Screening: From tornado diagrams to effective dimensions," European Journal of Operational Research, Elsevier, vol. 304(3), pages 1200-1211.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shi, Wen & Chen, Xi, 2019. "Controlled Morris method: A new factor screening approach empowered by a distribution-free sequential multiple testing procedure," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 299-314.
    2. Wen Shi & Xi Chen, 2018. "Efficient budget allocation strategies for elementary effects method in stochastic simulation," Naval Research Logistics (NRL), John Wiley & Sons, vol. 65(3), pages 218-241, April.
    3. Shi, Wen & Zhou, Qing & Zhou, Yanju, 2023. "An efficient elementary effect-based method for sensitivity analysis in identifying main and two-factor interaction effects," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    4. Kleijnen, Jack P.C., 2017. "Regression and Kriging metamodels with their experimental designs in simulation: A review," European Journal of Operational Research, Elsevier, vol. 256(1), pages 1-16.
    5. Shi, Wen & Kleijnen, Jack P.C. & Liu, Zhixue, 2014. "Factor screening for simulation with multiple responses: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 237(1), pages 136-147.
    6. Shi, Wen & Liu, Zhixue & Shang, Jennifer & Cui, Yujia, 2013. "Multi-criteria robust design of a JIT-based cross-docking distribution center for an auto parts supply chain," European Journal of Operational Research, Elsevier, vol. 229(3), pages 695-706.
    7. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
    8. T. Tony Cai & Wenguang Sun & Weinan Wang, 2019. "Covariate‐assisted ranking and screening for large‐scale two‐sample inference," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 81(2), pages 187-234, April.
    9. Nicola Rossi & Mario Bačić & Lovorka Librić & Meho Saša Kovačević, 2023. "Methodology for Identification of the Key Levee Parameters for Limit-State Analyses Based on Sequential Bifurcation," Sustainability, MDPI, vol. 15(6), pages 1-16, March.
    10. Alejandro Ochoa & John D Storey & Manuel Llinás & Mona Singh, 2015. "Beyond the E-Value: Stratified Statistics for Protein Domain Prediction," PLOS Computational Biology, Public Library of Science, vol. 11(11), pages 1-21, November.
    11. Chen, Xiongzhi, 2019. "Uniformly consistently estimating the proportion of false null hypotheses via Lebesgue–Stieltjes integral equations," Journal of Multivariate Analysis, Elsevier, vol. 173(C), pages 724-744.
    12. Cipolli III, William & Hanson, Timothy & McLain, Alexander C., 2016. "Bayesian nonparametric multiple testing," Computational Statistics & Data Analysis, Elsevier, vol. 101(C), pages 64-79.
    13. Guo Wenge & Peddada Shyamal, 2008. "Adaptive Choice of the Number of Bootstrap Samples in Large Scale Multiple Testing," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-21, March.
    14. Nik Tuzov & Frederi Viens, 2011. "Mutual fund performance: false discoveries, bias, and power," Annals of Finance, Springer, vol. 7(2), pages 137-169, May.
    15. Miecznikowski, Jeffrey C. & Gold, David & Shepherd, Lori & Liu, Song, 2011. "Deriving and comparing the distribution for the number of false positives in single step methods to control k-FWER," Statistics & Probability Letters, Elsevier, vol. 81(11), pages 1695-1705, November.
    16. Friguet, Chloé & Causeur, David, 2011. "Estimation of the proportion of true null hypotheses in high-dimensional data under dependence," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2665-2676, September.
    17. Zhao, Haibing & Fung, Wing Kam, 2016. "A powerful FDR control procedure for multiple hypotheses," Computational Statistics & Data Analysis, Elsevier, vol. 98(C), pages 60-70.
    18. Haibing Zhao & Wing Kam Fung, 2018. "Controlling mixed directional false discovery rate in multidimensional decisions with applications to microarray studies," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(2), pages 316-337, June.
    19. Ali Karimnezhad, 2022. "A simple yet efficient method of local false discovery rate estimation designed for genome-wide association data analysis," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 31(1), pages 159-180, March.
    20. Habiger, Joshua D. & Peña, Edsel A., 2014. "Compound p-value statistics for multiple testing procedures," Journal of Multivariate Analysis, Elsevier, vol. 126(C), pages 153-166.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:31:y:2019:i:4:p:745-770. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.