IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v53y2007i12p1916-1932.html
   My bibliography  Save this article

Selecting a Selection Procedure

Author

Listed:
  • Jürgen Branke

    (Institute AIFB, University of Karlsruhe, D-76128 Karlsruhe, Germany)

  • Stephen E. Chick

    (Technology and Operations Management, INSEAD, 77305 Fontainebleau Cedex, France)

  • Christian Schmidt

    (Institute AIFB, University of Karlsruhe, D-76128 Karlsruhe, Germany)

Abstract

Selection procedures are used in a variety of applications to select the best of a finite set of alternatives. "Best" is defined with respect to the largest mean, but the mean is inferred with statistical sampling, as in simulation optimization. There are a wide variety of procedures, which begs the question of which selection procedure to select. The main contribution of this paper is to identify, through extensive experimentation, the most effective selection procedures when samples are independent and normally distributed. We also (a) summarize the main structural approaches to deriving selection procedures, (b) formalize new sampling allocations and stopping rules, (c) identify strengths and weaknesses of the procedures, (d) identify some theoretical links between them, and (e) present an innovative empirical test bed with the most extensive numerical comparison of selection procedures to date. The most efficient and easiest to control procedures allocate samples with a Bayesian model for uncertainty about the means and use new adaptive stopping rules proposed here.

Suggested Citation

  • Jürgen Branke & Stephen E. Chick & Christian Schmidt, 2007. "Selecting a Selection Procedure," Management Science, INFORMS, vol. 53(12), pages 1916-1932, December.
    • Handle: RePEc:inm:ormnsc:v:53:y:2007:i:12:p:1916-1932
    DOI: 10.1287/mnsc.1070.0721
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.1070.0721
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.1070.0721?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Butler & Douglas J. Morrice & Peter W. Mullarkey, 2001. "A Multiple Attribute Utility Theory Approach to Ranking and Selection," Management Science, INFORMS, vol. 47(6), pages 800-816, June.
    2. Barry L. Nelson & David Goldsman, 2001. "Comparisons with a Standard in Simulation Experiments," Management Science, INFORMS, vol. 47(3), pages 449-463, March.
    3. Stephen E. Chick & Koichiro Inoue, 2002. "Corrigendum: New Selection Procedures," Operations Research, INFORMS, vol. 50(3), pages 566-566, June.
    4. David Goldsman & Seong-Hee Kim & William S. Marshall & Barry L. Nelson, 2002. "Ranking and Selection for Steady-State Simulation: Procedures and Perspectives," INFORMS Journal on Computing, INFORMS, vol. 14(1), pages 2-19, February.
    5. Chen, E. Jack & Kelton, W. David, 2005. "Sequential selection procedures: Using sample means to improve efficiency," European Journal of Operational Research, Elsevier, vol. 166(1), pages 133-153, October.
    6. Justin Boesel & Barry L. Nelson & Seong-Hee Kim, 2003. "Using Ranking and Selection to “Clean Up” after Simulation Optimization," Operations Research, INFORMS, vol. 51(5), pages 814-825, October.
    7. Michael C. Fu & Jian-Qiang Hu & Chun-Hung Chen & Xiaoping Xiong, 2007. "Simulation Allocation for Determining the Best Design in the Presence of Correlated Sampling," INFORMS Journal on Computing, INFORMS, vol. 19(1), pages 101-111, February.
    8. Stephen E. Chick & Yaozhong Wu, 2005. "Selection Procedures with Frequentist Expected Opportunity Cost Bounds," Operations Research, INFORMS, vol. 53(5), pages 867-878, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yoon, Moonyoung & Bekker, James, 2019. "Considering sample means in Rinott’s procedure with a Bayesian approach," European Journal of Operational Research, Elsevier, vol. 273(1), pages 249-258.
    2. Batur, D. & Choobineh, F., 2010. "A quantile-based approach to system selection," European Journal of Operational Research, Elsevier, vol. 202(3), pages 764-772, May.
    3. Groves, Matthew & Branke, Juergen, 2019. "Top-κ selection with pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 274(2), pages 615-626.
    4. Wang, Tianxiang & Xu, Jie & Hu, Jian-Qiang & Chen, Chun-Hung, 2023. "Efficient estimation of a risk measure requiring two-stage simulation optimization," European Journal of Operational Research, Elsevier, vol. 305(3), pages 1355-1365.
    5. Tsai, Shing Chih & Fu, Sheng Yang, 2014. "Genetic-algorithm-based simulation optimization considering a single stochastic constraint," European Journal of Operational Research, Elsevier, vol. 236(1), pages 113-125.
    6. Fei, Xin & Gülpınar, Nalân & Branke, Jürgen, 2019. "Efficient solution selection for two-stage stochastic programs," European Journal of Operational Research, Elsevier, vol. 277(3), pages 918-929.
    7. Tan, Kim Hua & Noble, James & Sato, Yuji & Tse, Ying Kei, 2011. "A marginal analysis guided technology evaluation and selection," International Journal of Production Economics, Elsevier, vol. 131(1), pages 15-21, May.
    8. Kleijnen, Jack P.C. & Beers, Wim van & Nieuwenhuyse, Inneke van, 2010. "Constrained optimization in expensive simulation: Novel approach," European Journal of Operational Research, Elsevier, vol. 202(1), pages 164-174, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Juergen Branke & Wen Zhang, 2019. "Identifying efficient solutions via simulation: myopic multi-objective budget allocation for the bi-objective case," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 41(3), pages 831-865, September.
    2. Demet Batur & Lina Wang & F. Fred Choobineh, 2018. "Methods for System Selection Based on Sequential Mean–Variance Analysis," INFORMS Journal on Computing, INFORMS, vol. 30(4), pages 724-738, November.
    3. Yoon, Moonyoung & Bekker, James, 2019. "Considering sample means in Rinott’s procedure with a Bayesian approach," European Journal of Operational Research, Elsevier, vol. 273(1), pages 249-258.
    4. Lee, Loo Hay & Chew, Ek Peng & Manikam, Puvaneswari, 2006. "A general framework on the simulation-based optimization under fixed computing budget," European Journal of Operational Research, Elsevier, vol. 174(3), pages 1828-1841, November.
    5. Ahmed, Mohamed A. & Alkhamis, Talal M., 2009. "Simulation optimization for an emergency department healthcare unit in Kuwait," European Journal of Operational Research, Elsevier, vol. 198(3), pages 936-942, November.
    6. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    7. Stephen E. Chick & Noah Gans, 2009. "Economic Analysis of Simulation Selection Problems," Management Science, INFORMS, vol. 55(3), pages 421-437, March.
    8. Jeffrey W. Herrmann & Kunal Mehta, 2020. "Lookahead and Hybrid Sample Allocation Procedures for Multiple Attribute Selection Decisions," Papers 2007.16119, arXiv.org.
    9. Kleijnen, J.P.C. & Sanchez, S.M. & Lucas, T.W. & Cioppa, T.M., 2003. "A User's Guide to the Brave New World of Designing Simulation Experiments," Discussion Paper 2003-1, Tilburg University, Center for Economic Research.
    10. Teng, Suyan & Lee, Loo Hay & Chew, Ek Peng, 2010. "Integration of indifference-zone with multi-objective computing budget allocation," European Journal of Operational Research, Elsevier, vol. 203(2), pages 419-429, June.
    11. Tsai, Shing Chih & Chen, Sin Ting, 2017. "A simulation-based multi-objective optimization framework: A case study on inventory management," Omega, Elsevier, vol. 70(C), pages 148-159.
    12. Shing Chih Tsai & Tse Yang, 2017. "Rapid screening algorithms for stochastically constrained problems," Annals of Operations Research, Springer, vol. 254(1), pages 425-447, July.
    13. Kabirian, Alireza & Ólafsson, Sigurdur, 2011. "Continuous optimization via simulation using Golden Region search," European Journal of Operational Research, Elsevier, vol. 208(1), pages 19-27, January.
    14. L. Jeff Hong & Jun Luo & Barry L. Nelson, 2015. "Chance Constrained Selection of the Best," INFORMS Journal on Computing, INFORMS, vol. 27(2), pages 317-334, May.
    15. Healey, Christopher M. & Andradóttir, Sigrún & Kim, Seong-Hee, 2013. "Efficient comparison of constrained systems using dormancy," European Journal of Operational Research, Elsevier, vol. 224(2), pages 340-352.
    16. Shing Chih Tsai, 2013. "Rapid Screening Procedures for Zero-One Optimization via Simulation," INFORMS Journal on Computing, INFORMS, vol. 25(2), pages 317-331, May.
    17. Jason R. W. Merrick, 2009. "Bayesian Simulation and Decision Analysis: An Expository Survey," Decision Analysis, INFORMS, vol. 6(4), pages 222-238, December.
    18. Batur, D. & Choobineh, F., 2010. "A quantile-based approach to system selection," European Journal of Operational Research, Elsevier, vol. 202(3), pages 764-772, May.
    19. Stephen E. Chick & Jürgen Branke & Christian Schmidt, 2010. "Sequential Sampling to Myopically Maximize the Expected Value of Information," INFORMS Journal on Computing, INFORMS, vol. 22(1), pages 71-80, February.
    20. Vadim Lesnevski & Barry L. Nelson & Jeremy Staum, 2007. "Simulation of Coherent Risk Measures Based on Generalized Scenarios," Management Science, INFORMS, vol. 53(11), pages 1756-1769, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:53:y:2007:i:12:p:1916-1932. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.