IDEAS home Printed from https://ideas.repec.org/a/inm/oropre/v63y2015i4p868-876.html
   My bibliography  Save this article

The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples

Author

Listed:
  • Dimitris Bertsimas

    (Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • Mac Johnson

    (Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

  • Nathan Kallus

    (Operations Research Center, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139)

Abstract

Random assignment, typically seen as the standard in controlled trials, aims to make experimental groups statistically equivalent before treatment. However, with a small sample, which is a practical reality in many disciplines, randomized groups are often too dissimilar to be useful. We propose an approach based on discrete linear optimization to create groups whose discrepancy in their means and variances is several orders of magnitude smaller than with randomization. We provide theoretical and computational evidence that groups created by optimization have exponentially lower discrepancy than those created by randomization and that this allows for more powerful statistical inference.

Suggested Citation

  • Dimitris Bertsimas & Mac Johnson & Nathan Kallus, 2015. "The Power of Optimization Over Randomization in Designing Experiments Involving Small Samples," Operations Research, INFORMS, vol. 63(4), pages 868-876, August.
    • Handle: RePEc:inm:oropre:v:63:y:2015:i:4:p:868-876
    DOI: 10.1287/opre.2015.1361
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/opre.2015.1361
    Download Restriction: no
    File URL: https://libkey.io/10.1287/opre.2015.1361?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Morris, Carl, 1979. "A finite selection model for experimental design of the health insurance study," Journal of Econometrics, Elsevier, vol. 11(1), pages 43-61, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jason J. Sauppe & Sheldon H. Jacobson, 2017. "The role of covariate balance in observational studies," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(4), pages 323-344, June.
    2. Belmiro P. M. Duarte & Anthony C. Atkinson & David Pedrosa & Marlena van Munster, 2024. "Compound Optimum Designs for Clinical Trials in Personalized Medicine," Mathematics, MDPI, vol. 12(19), pages 1-20, September.
    3. Nikhil Bhat & Vivek F. Farias & Ciamac C. Moallemi & Deeksha Sinha, 2020. "Near-Optimal A-B Testing," Management Science, INFORMS, vol. 66(10), pages 4477-4495, October.
    4. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    5. Jinglong Zhao & Zijie Zhou, 2022. "Pigeonhole Design: Balancing Sequential Experiments from an Online Matching Perspective," Papers 2201.12936, arXiv.org, revised May 2024.
    6. Ruoxuan Xiong & Susan Athey & Mohsen Bayati & Guido Imbens, 2024. "Optimal Experimental Design for Staggered Rollouts," Management Science, INFORMS, vol. 70(8), pages 5317-5336, August.
    7. Guiteras, Raymond P. & Levine, David I. & Polley, Thomas H., 2016. "The pursuit of balance in sequential randomized trials," Development Engineering, Elsevier, vol. 1(C), pages 12-25.
    8. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    9. Dimitris Bertsimas & Bradley Sturt, 2020. "Computation of Exact Bootstrap Confidence Intervals: Complexity and Deterministic Algorithms," Operations Research, INFORMS, vol. 68(3), pages 949-964, May.
    10. Martin Cousineau & Vedat Verter & Susan A. Murphy & Joelle Pineau, 2022. "Estimating causal effects with optimization-based methods: A review and empirical comparison," Papers 2203.00097, arXiv.org.
    11. Rauf Ahmad & Per Johansson & Mårten Schultzberg, 2024. "Is Fisher inference inferior to Neyman inference for policy analysis?," Statistical Papers, Springer, vol. 65(6), pages 3425-3445, August.
    12. Mogues, Tewodaj & Van Campenhout, Bjorn & Miehe, Caroline & Kabunga, Nassul, 2023. "The impact of community-based monitoring on public service delivery: A randomized control trial in Uganda," World Development, Elsevier, vol. 172(C).
    13. Qiong Zhang & Amin Khademi & Yongjia Song, 2022. "Min-Max Optimal Design of Two-Armed Trials with Side Information," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 165-182, January.
    14. Dimitris Bertsimas & Nikita Korolko & Alexander M. Weinstein, 2019. "Covariate-Adaptive Optimization in Online Clinical Trials," Operations Research, INFORMS, vol. 67(4), pages 1150-1161, July.
    15. Cousineau, Martin & Verter, Vedat & Murphy, Susan A. & Pineau, Joelle, 2023. "Estimating causal effects with optimization-based methods: A review and empirical comparison," European Journal of Operational Research, Elsevier, vol. 304(2), pages 367-380.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Deb, Partha & Trivedi, Pravin K., 2002. "The structure of demand for health care: latent class versus two-part models," Journal of Health Economics, Elsevier, vol. 21(4), pages 601-625, July.
    2. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    3. Aviva Aron-Dine & Liran Einav & Amy Finkelstein, 2013. "The RAND Health Insurance Experiment, Three Decades Later," Journal of Economic Perspectives, American Economic Association, vol. 27(1), pages 197-222, Winter.
    4. Connelly, Luke B., 2003. "Balancing the Number and Size of Sites: An Economic Approach to the Optimal Design of Cluster Samples," MPRA Paper 14676, University Library of Munich, Germany.
    5. Jeffrey E. Harris, 1985. "Macroexperiments versus Microexperiments for Health Policy," NBER Chapters, in: Social Experimentation, pages 145-186, National Bureau of Economic Research, Inc.
    6. Haizhen Lin & Daniel W. Sacks, 2016. "Intertemporal Substitution in Health Care Demand: Evidence from the RAND Health Insurance Experiment," NBER Working Papers 22802, National Bureau of Economic Research, Inc.
    7. Cecilia S. Diaz-Campo, M. Antonella Mancino, 2023. "What We RANDomly Did Not Learn: Opioid Elasticities and Underlying Mechanisms," LCERPA Working Papers bm0139, Laurier Centre for Economic Research and Policy Analysis.
    8. Grant, William C. & Anstrom, Kevin J., 2008. "Minimizing selection bias in randomized trials: A Nash equilibrium approach to optimal randomization," Journal of Economic Behavior & Organization, Elsevier, vol. 66(3-4), pages 606-624, June.
    9. Lin, Haizhen & Sacks, Daniel W., 2019. "Intertemporal substitution in health care demand: Evidence from the RAND Health Insurance Experiment," Journal of Public Economics, Elsevier, vol. 175(C), pages 29-43.
    10. Dimitris Bertsimas & Nikita Korolko & Alexander M. Weinstein, 2019. "Covariate-Adaptive Optimization in Online Clinical Trials," Operations Research, INFORMS, vol. 67(4), pages 1150-1161, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:oropre:v:63:y:2015:i:4:p:868-876. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.