IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v52y2008i12p5215-5228.html
   My bibliography  Save this article

Pooled ANOVA

Author

Listed:
  • Last, Michael
  • Luta, Gheorghe
  • Orso, Alex
  • Porter, Adam
  • Young, Stan

Abstract

We introduce Pooled ANOVA, a greedy algorithm to sequentially select the rare important factors from a large set of factors. Problems such as computer simulations and software performance tuning involve a large number of factors, few of which have an important effect on the outcome or performance measure. We pool multiple factors together, and test the pool for significance. If the pool has a significant effect we retain the factors for deconfounding. If not, we either declare that none of the factors are important, or retain them for follow-up decoding, depending on our assumptions and stage of testing. The sparser important factors are, the bigger the savings. Pooled ANOVA requires fewer assumptions than other, similar methods (e.g. sequential bifurcation), such as not requiring all important effects to have the same sign. We demonstrate savings of 25%-35% when compared to a conventional ANOVA, and also the ability to work in a setting where Sequential Bifurcation fails.

Suggested Citation

  • Last, Michael & Luta, Gheorghe & Orso, Alex & Porter, Adam & Young, Stan, 2008. "Pooled ANOVA," Computational Statistics & Data Analysis, Elsevier, vol. 52(12), pages 5215-5228, August.
    • Handle: RePEc:eee:csdana:v:52:y:2008:i:12:p:5215-5228
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(08)00216-8
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kleijnen, J.P.C. & Bettonvil, B.W.M., 1997. "Searching for important factors in simulation models with many factors : Sequential bifurcation," Other publications TiSEM be826993-22f9-4cb3-89df-3, Tilburg University, School of Economics and Management.
    2. S. M. Lewis & A. M. Dean, 2001. "Detection of interactions in experiments on large numbers of factors," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(4), pages 633-672.
    3. Bettonvil, Bert & Kleijnen, Jack P. C., 1997. "Searching for important factors in simulation models with many factors: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 96(1), pages 180-194, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kleijnen, J.P.C., 2007. "Screening Experiments for Simulation : A Review," Discussion Paper 2007-21, Tilburg University, Center for Economic Research.
    2. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2006. "Controlled Sequential Bifurcation: A New Factor-Screening Method for Discrete-Event Simulation," Operations Research, INFORMS, vol. 54(4), pages 743-755, August.
    3. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    4. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    5. Van Groenendaal, Willem J. H. & Kleijnen, Jack P. C., 2002. "Deterministic versus stochastic sensitivity analysis in investment problems: An environmental case study," European Journal of Operational Research, Elsevier, vol. 141(1), pages 8-20, August.
    6. Kleijnen, Jack P. C. & Sargent, Robert G., 2000. "A methodology for fitting and validating metamodels in simulation," European Journal of Operational Research, Elsevier, vol. 120(1), pages 14-29, January.
    7. Kleijnen, J.P.C. & Sanchez, S.M. & Lucas, T.W. & Cioppa, T.M., 2003. "A User's Guide to the Brave New World of Designing Simulation Experiments," Discussion Paper 2003-1, Tilburg University, Center for Economic Research.
    8. Shi, Wen & Kleijnen, J.P.C., 2017. "Testing the Assumptions of Sequential Bifurcation for Factor Screening (revision of CentER DP 2015-034)," Other publications TiSEM 763fd6f8-b618-4b06-a284-5, Tilburg University, School of Economics and Management.
    9. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    10. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
    11. Kleijnen, J.P.C., 2009. "Sensitivity Analysis of Simulation Models," Discussion Paper 2009-11, Tilburg University, Center for Economic Research.
    12. Shi, Wen & Kleijnen, Jack P.C. & Liu, Zhixue, 2014. "Factor screening for simulation with multiple responses: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 237(1), pages 136-147.
    13. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    14. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2010. "Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 482-492, August.
    15. Shi, Wen & Chen, Xi, 2019. "Controlled Morris method: A new factor screening approach empowered by a distribution-free sequential multiple testing procedure," Reliability Engineering and System Safety, Elsevier, vol. 189(C), pages 299-314.
    16. Singh, Ramkishore & Lazarus, I.J. & Kishore, V.V.N., 2016. "Uncertainty and sensitivity analyses of energy and visual performances of office building with external venetian blind shading in hot-dry climate," Applied Energy, Elsevier, vol. 184(C), pages 155-170.
    17. Shi, W. & Kleijnen, J.P.C., 2015. "Validating the Assumptions of Sequential Bifurcation in Factor Screening," Discussion Paper 2015-034, Tilburg University, Center for Economic Research.
    18. Batmaz, Inci & Tunali, Semra, 2003. "Small response surface designs for metamodel estimation," European Journal of Operational Research, Elsevier, vol. 145(2), pages 455-470, March.
    19. Elmar Plischke & Emanuele Borgonovo, 2020. "Fighting the Curse of Sparsity: Probabilistic Sensitivity Measures From Cumulative Distribution Functions," Risk Analysis, John Wiley & Sons, vol. 40(12), pages 2639-2660, December.
    20. Kleijnen, J.P.C., 1997. "Experimental Design for Sensitivity Analysis, Optimization and Validation of Simulation Models," Discussion Paper 1997-52, Tilburg University, Center for Economic Research.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:52:y:2008:i:12:p:5215-5228. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.