IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v189y2019icp299-314.html
   My bibliography  Save this article

Controlled Morris method: A new factor screening approach empowered by a distribution-free sequential multiple testing procedure

Author

Listed:
  • Shi, Wen
  • Chen, Xi

Abstract

The Morris method (MM) is a well known model-free factor screening approach that is considered particularly effective when the number of factors is large or when the computer model is computationally expensive to run. In this paper, we propose the controlled Morris method (CMM) for simulation-based factor screening, which integrates a distribution-free sequential multiple testing procedure with MM to control the Type I and Type II familywise error rates at the prescribed levels while achieving a high computational efficiency. Numerical experiments are provided to demonstrate the efficacy and efficiency of CMM.

Suggested Citation

  • Shi, Wen & Chen, Xi, 2019. "Controlled Morris method: A new factor screening approach empowered by a distribution-free sequential multiple testing procedure," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 299-314.
    • Handle: RePEc:eee:reensy:v:189:y:2019:i:c:p:299-314
    DOI: 10.1016/j.ress.2019.04.038
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832018315242
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2019.04.038?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    2. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    3. Ge, Qiao & Ciuffo, Biagio & Menendez, Monica, 2015. "Combining screening and metamodel-based methods: An efficient sequential approach for the sensitivity analysis of model outputs," Reliability Engineering and System Safety, Elsevier, vol. 134(C), pages 334-344.
    4. Pujol, Gilles, 2009. "Simplex-based screening designs for estimating metamodels," Reliability Engineering and System Safety, Elsevier, vol. 94(7), pages 1156-1160.
    5. Jack P.C. Kleijnen, 2015. "Design and Analysis of Simulation Experiments," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-18087-8, September.
    6. Shi, Wen & Kleijnen, Jack P.C. & Liu, Zhixue, 2014. "Factor screening for simulation with multiple responses: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 237(1), pages 136-147.
    7. Ge, Qiao & Menendez, Monica, 2017. "Extending Morris method for qualitative global sensitivity analysis of models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 162(C), pages 28-39.
    8. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2006. "Controlled Sequential Bifurcation: A New Factor-Screening Method for Discrete-Event Simulation," Operations Research, INFORMS, vol. 54(4), pages 743-755, August.
    9. Sanchez, Susan M. & Moeeni, Farhad & Sanchez, Paul J., 2006. "So many factors, so little time...Simulation experiments in the frequency domain," International Journal of Production Economics, Elsevier, vol. 103(1), pages 149-165, September.
    10. Panos Parpas & Berk Ustun & Mort Webster & Quang Kha Tran, 2015. "Importance Sampling in Stochastic Programming: A Markov Chain Monte Carlo Approach," INFORMS Journal on Computing, INFORMS, vol. 27(2), pages 358-377, May.
    11. Xiao, Sinan & Lu, Zhenzhou & Xu, Liyang, 2016. "A new effective screening design for structural sensitivity analysis of failure probability with the epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 156(C), pages 1-14.
    12. Deman, G. & Kerrou, J. & Benabderrahmane, H. & Perrochet, P., 2015. "Sensitivity analysis of groundwater lifetime expectancy to hydro-dispersive parameters: The case of ANDRA Meuse/Haute-Marne site," Reliability Engineering and System Safety, Elsevier, vol. 134(C), pages 276-286.
    13. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2010. "Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 482-492, August.
    14. Guohui Zhang & Yinhai Wang, 2014. "A Gaussian Kernel-Based Approach for Modeling Vehicle Headway Distributions," Transportation Science, INFORMS, vol. 48(2), pages 206-216, May.
    15. Kleijnen, J.P.C. & Bettonvil, B.W.M., 1997. "Searching for important factors in simulation models with many factors : Sequential bifurcation," Other publications TiSEM be826993-22f9-4cb3-89df-3, Tilburg University, School of Economics and Management.
    16. Morris, Max D., 2006. "Input screening: Finding the important model inputs on a budget," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1252-1256.
    17. Lamoureux, Benjamin & Mechbal, Nazih & Massé, Jean-Rémi, 2014. "A combined sensitivity analysis and kriging surrogate modeling for early validation of health indicators," Reliability Engineering and System Safety, Elsevier, vol. 130(C), pages 12-26.
    18. Bettonvil, Bert & Kleijnen, Jack P. C., 1997. "Searching for important factors in simulation models with many factors: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 96(1), pages 180-194, January.
    19. Dunson, David B., 2018. "Statistics in the big data era: Failures of the machine," Statistics & Probability Letters, Elsevier, vol. 136(C), pages 4-9.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shi, Wen & Zhou, Qing & Zhou, Yanju, 2023. "An efficient elementary effect-based method for sensitivity analysis in identifying main and two-factor interaction effects," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    2. Borgonovo, Emanuele & Rabitti, Giovanni, 2023. "Screening: From tornado diagrams to effective dimensions," European Journal of Operational Research, Elsevier, vol. 304(3), pages 1200-1211.
    3. Yogesh P. Khare & Rajendra Paudel & Ruscena Wiederholt & Anteneh Z. Abiy & Thomas Van Lent & Stephen E. Davis & Younggu Her, 2021. "Watershed Response to Legacy Phosphorus and Best Management Practices in an Impacted Agricultural Watershed in Florida, U.S.A," Land, MDPI, vol. 10(9), pages 1-22, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    2. Wen Shi & Xi Chen, 2018. "Efficient budget allocation strategies for elementary effects method in stochastic simulation," Naval Research Logistics (NRL), John Wiley & Sons, vol. 65(3), pages 218-241, April.
    3. Shi, Wen & Zhou, Qing & Zhou, Yanju, 2023. "An efficient elementary effect-based method for sensitivity analysis in identifying main and two-factor interaction effects," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    4. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
    5. Shi, Wen & Kleijnen, Jack P.C. & Liu, Zhixue, 2014. "Factor screening for simulation with multiple responses: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 237(1), pages 136-147.
    6. Shi, Wen & Kleijnen, J.P.C., 2017. "Testing the Assumptions of Sequential Bifurcation for Factor Screening (revision of CentER DP 2015-034)," Other publications TiSEM 763fd6f8-b618-4b06-a284-5, Tilburg University, School of Economics and Management.
    7. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    8. Kleijnen, J.P.C., 2009. "Sensitivity Analysis of Simulation Models," Discussion Paper 2009-11, Tilburg University, Center for Economic Research.
    9. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    10. Shi, W. & Kleijnen, J.P.C., 2015. "Validating the Assumptions of Sequential Bifurcation in Factor Screening," Discussion Paper 2015-034, Tilburg University, Center for Economic Research.
    11. Nicola Rossi & Mario Bačić & Lovorka Librić & Meho Saša Kovačević, 2023. "Methodology for Identification of the Key Levee Parameters for Limit-State Analyses Based on Sequential Bifurcation," Sustainability, MDPI, vol. 15(6), pages 1-16, March.
    12. Shi, Wen & Liu, Zhixue & Shang, Jennifer & Cui, Yujia, 2013. "Multi-criteria robust design of a JIT-based cross-docking distribution center for an auto parts supply chain," European Journal of Operational Research, Elsevier, vol. 229(3), pages 695-706.
    13. Kleijnen, Jack P.C., 2017. "Regression and Kriging metamodels with their experimental designs in simulation: A review," European Journal of Operational Research, Elsevier, vol. 256(1), pages 1-16.
    14. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2010. "Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 482-492, August.
    15. Kleijnen, J.P.C., 2007. "Screening Experiments for Simulation : A Review," Discussion Paper 2007-21, Tilburg University, Center for Economic Research.
    16. Shen, Hua & Wan, Hong, 2009. "Controlled sequential factorial design for simulation factor screening," European Journal of Operational Research, Elsevier, vol. 198(2), pages 511-519, October.
    17. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    18. Dellino, Gabriella & Kleijnen, Jack P.C. & Meloni, Carlo, 2010. "Robust optimization in simulation: Taguchi and Response Surface Methodology," International Journal of Production Economics, Elsevier, vol. 125(1), pages 52-59, May.
    19. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    20. Van Groenendaal, Willem J. H. & Kleijnen, Jack P. C., 2002. "Deterministic versus stochastic sensitivity analysis in investment problems: An environmental case study," European Journal of Operational Research, Elsevier, vol. 141(1), pages 8-20, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:189:y:2019:i:c:p:299-314. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.