IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v202y2010i3p764-772.html
   My bibliography  Save this article

A quantile-based approach to system selection

Author

Listed:
  • Batur, D.
  • Choobineh, F.

Abstract

We propose a quantile-based ranking and selection (R&S) procedure for comparing a finite set of stochastic systems via simulation. Our R&S procedure uses a quantile set of the simulated probability distribution of a performance characteristic of interest that best represents the most appropriate selection criterion as the basis for comparison. Since this quantile set may represent either the downside risk, upside risk, or central tendency of the performance characteristic, the proposed approach is more flexible than the traditional mean-based approach to R&S. We first present a procedure that selects the best system from among K systems, and then we modified that procedure for the case where KÂ -Â 1 systems are compared against a standard system. We present a set of experiments to highlight the flexibility of the proposed procedures.

Suggested Citation

  • Batur, D. & Choobineh, F., 2010. "A quantile-based approach to system selection," European Journal of Operational Research, Elsevier, vol. 202(3), pages 764-772, May.
    • Handle: RePEc:eee:ejores:v:202:y:2010:i:3:p:764-772
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377-2217(09)00385-3
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Barry L. Nelson & David Goldsman, 2001. "Comparisons with a Standard in Simulation Experiments," Management Science, INFORMS, vol. 47(3), pages 449-463, March.
    2. P. Heidelberger & P. A. W. Lewis, 1984. "Quantile Estimation in Dependent Sequences," Operations Research, INFORMS, vol. 32(1), pages 185-209, February.
    3. Stephen E. Chick & Koichiro Inoue, 2001. "New Two-Stage and Sequential Procedures for Selecting the Best Simulated System," Operations Research, INFORMS, vol. 49(5), pages 732-743, October.
    4. Pichitlamken, Juta & Nelson, Barry L. & Hong, L. Jeff, 2006. "A sequential procedure for neighborhood selection-of-the-best in optimization via simulation," European Journal of Operational Research, Elsevier, vol. 173(1), pages 283-298, August.
    5. Chen, E. Jack & Kelton, W. David, 2006. "Quantile and tolerance-interval estimation in simulation," European Journal of Operational Research, Elsevier, vol. 168(2), pages 520-540, January.
    6. Jürgen Branke & Stephen E. Chick & Christian Schmidt, 2007. "Selecting a Selection Procedure," Management Science, INFORMS, vol. 53(12), pages 1916-1932, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Tsai, Shing Chih & Chu, I-Hao, 2012. "Controlled multistage selection procedures for comparison with a standard," European Journal of Operational Research, Elsevier, vol. 223(3), pages 709-721.
    2. Chang, Kuo-Hao, 2015. "A direct search method for unconstrained quantile-based simulation optimization," European Journal of Operational Research, Elsevier, vol. 246(2), pages 487-495.
    3. Zhongshun Shi & Yijie Peng & Leyuan Shi & Chun-Hung Chen & Michael C. Fu, 2022. "Dynamic Sampling Allocation Under Finite Simulation Budget for Feasibility Determination," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 557-568, January.
    4. Meloni, Carlo & Pranzo, Marco & Samà, Marcella, 2022. "Evaluation of VaR and CVaR for the makespan in interval valued blocking job shops," International Journal of Production Economics, Elsevier, vol. 247(C).
    5. Demet Batur & F. Fred Choobineh, 2021. "Selecting the Best Alternative Based on Its Quantile," INFORMS Journal on Computing, INFORMS, vol. 33(2), pages 657-671, May.
    6. Kleijnen, Jack P.C. & Pierreval, Henri & Zhang, Jin, 2011. "Methodology for determining the acceptability of system designs in uncertain environments," European Journal of Operational Research, Elsevier, vol. 209(2), pages 176-183, March.
    7. Yijie Peng & Chun-Hung Chen & Michael C. Fu & Jian-Qiang Hu & Ilya O. Ryzhov, 2021. "Efficient Sampling Allocation Procedures for Optimal Quantile Selection," INFORMS Journal on Computing, INFORMS, vol. 33(1), pages 230-245, January.
    8. Gabriella Dellino & Jack P. C. Kleijnen & Carlo Meloni, 2012. "Robust Optimization in Simulation: Taguchi and Krige Combined," INFORMS Journal on Computing, INFORMS, vol. 24(3), pages 471-484, August.
    9. J P C Kleijnen & W C M van Beers, 2013. "Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 64(5), pages 708-717, May.
    10. Cheng, Zhenxia & Luo, Jun & Wu, Ruijing, 2023. "On the finite-sample statistical validity of adaptive fully sequential procedures," European Journal of Operational Research, Elsevier, vol. 307(1), pages 266-278.
    11. Batur, D. & Choobineh, F., 2012. "Stochastic dominance based comparison for system selection," European Journal of Operational Research, Elsevier, vol. 220(3), pages 661-672.
    12. Dongwook Shin & Mark Broadie & Assaf Zeevi, 2022. "Practical Nonparametric Sampling Strategies for Quantile-Based Ordinal Optimization," INFORMS Journal on Computing, INFORMS, vol. 34(2), pages 752-768, March.
    13. Saurabh Bansal & Genaro J. Gutierrez, 2020. "Estimating Uncertainties Using Judgmental Forecasts with Expert Heterogeneity," Operations Research, INFORMS, vol. 68(2), pages 363-380, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Batur, D. & Choobineh, F., 2012. "Stochastic dominance based comparison for system selection," European Journal of Operational Research, Elsevier, vol. 220(3), pages 661-672.
    2. Demet Batur & F. Fred Choobineh, 2021. "Selecting the Best Alternative Based on Its Quantile," INFORMS Journal on Computing, INFORMS, vol. 33(2), pages 657-671, May.
    3. Christos Alexopoulos & David Goldsman & Anup C. Mokashi & Kai-Wen Tien & James R. Wilson, 2019. "Sequest: A Sequential Procedure for Estimating Quantiles in Steady-State Simulations," Operations Research, INFORMS, vol. 67(4), pages 1162-1183, July.
    4. Demet Batur & Lina Wang & F. Fred Choobineh, 2018. "Methods for System Selection Based on Sequential Mean–Variance Analysis," INFORMS Journal on Computing, INFORMS, vol. 30(4), pages 724-738, November.
    5. Zhongshun Shi & Yijie Peng & Leyuan Shi & Chun-Hung Chen & Michael C. Fu, 2022. "Dynamic Sampling Allocation Under Finite Simulation Budget for Feasibility Determination," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 557-568, January.
    6. Weiwei Fan & L. Jeff Hong & Barry L. Nelson, 2016. "Indifference-Zone-Free Selection of the Best," Operations Research, INFORMS, vol. 64(6), pages 1499-1514, December.
    7. Stephen E. Chick & Noah Gans, 2009. "Economic Analysis of Simulation Selection Problems," Management Science, INFORMS, vol. 55(3), pages 421-437, March.
    8. Shing Chih Tsai & Tse Yang, 2017. "Rapid screening algorithms for stochastically constrained problems," Annals of Operations Research, Springer, vol. 254(1), pages 425-447, July.
    9. Tsai, Shing Chih & Chu, I-Hao, 2012. "Controlled multistage selection procedures for comparison with a standard," European Journal of Operational Research, Elsevier, vol. 223(3), pages 709-721.
    10. Hyeong Soo Chang & Jiaqiao Hu, 2012. "On the Probability of Correct Selection in Ordinal Comparison over Dynamic Networks," Journal of Optimization Theory and Applications, Springer, vol. 155(2), pages 594-604, November.
    11. Kabirian, Alireza & Ólafsson, Sigurdur, 2011. "Continuous optimization via simulation using Golden Region search," European Journal of Operational Research, Elsevier, vol. 208(1), pages 19-27, January.
    12. Justin Boesel & Barry L. Nelson & Seong-Hee Kim, 2003. "Using Ranking and Selection to “Clean Up” after Simulation Optimization," Operations Research, INFORMS, vol. 51(5), pages 814-825, October.
    13. Guangwu Liu & Liu Jeff Hong, 2009. "Kernel estimation of quantile sensitivities," Naval Research Logistics (NRL), John Wiley & Sons, vol. 56(6), pages 511-525, September.
    14. Lee, Loo Hay & Chew, Ek Peng & Manikam, Puvaneswari, 2006. "A general framework on the simulation-based optimization under fixed computing budget," European Journal of Operational Research, Elsevier, vol. 174(3), pages 1828-1841, November.
    15. Mingbin Ben Feng & Eunhye Song, 2020. "Optimal Nested Simulation Experiment Design via Likelihood Ratio Method," Papers 2008.13087, arXiv.org, revised Jul 2021.
    16. Jürgen Branke & Stephen E. Chick & Christian Schmidt, 2007. "Selecting a Selection Procedure," Management Science, INFORMS, vol. 53(12), pages 1916-1932, December.
    17. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    18. Sigrún Andradóttir & Andrei A. Prudius, 2009. "Balanced Explorative and Exploitative Search with Estimation for Simulation Optimization," INFORMS Journal on Computing, INFORMS, vol. 21(2), pages 193-208, May.
    19. Kleijnen, J.P.C. & Sanchez, S.M. & Lucas, T.W. & Cioppa, T.M., 2003. "A User's Guide to the Brave New World of Designing Simulation Experiments," Discussion Paper 2003-1, Tilburg University, Center for Economic Research.
    20. Groves, Matthew & Branke, Juergen, 2019. "Top-κ selection with pairwise comparisons," European Journal of Operational Research, Elsevier, vol. 274(2), pages 615-626.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:202:y:2010:i:3:p:764-772. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.