IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/6278.html
   My bibliography  Save this paper

Perils of simulation : parallel streams and the case of stata's rnormal command

Author

Listed:
  • Ozier, Owen

Abstract

Large-scale simulation-based studies rely on at least three properties of pseudorandom number sequences: they behave in many ways like truly random numbers; they can be replicated; and they can be generated in parallel. There has been some divergence, however, between empirical techniques employing random numbers, and the standard battery of tests used to validate them. A random number generator that passes tests for any single stream of random numbers may fail the same tests when it is used to generate multiple streams in parallel. The lack of systematic testing of parallel streams leaves statistical software with important potential vulnerabilities. This paper shows one such vulnerability in Stata's rnormal function that went unnoticed for almost four years, and how to detect it. It then shows practical implications for the use of parallel streams in existing software.

Suggested Citation

  • Ozier, Owen, 2012. "Perils of simulation : parallel streams and the case of stata's rnormal command," Policy Research Working Paper Series 6278, The World Bank.
    • Handle: RePEc:wbk:wbrwps:6278
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2012/11/28/000158349_20121128134827/Rendered/PDF/wps6278.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.
    2. Nigel Smeeton & Nicholas J. Cox, 2003. "Do-it-yourself shuffling and the number of runs under randomness," Stata Journal, StataCorp LP, vol. 3(3), pages 270-277, September.
    3. B. D. McCullough, 2006. "A review of TESTU01," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(5), pages 677-682, July.
    4. Tirler, Gunter & Dalgaard, Peter & Hormann, Wolfgang & Leydold, Josef, 2004. "An error in the Kinderman-Ramage method and how to fix it," Computational Statistics & Data Analysis, Elsevier, vol. 47(3), pages 433-440, October.
    5. Busso, Matias & DiNardo, John & McCrary, Justin, 2009. "New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators," IZA Discussion Papers 3998, Institute of Labor Economics (IZA).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yalta, A. Talha, 2007. "The Numerical Reliability of GAUSS 8.0," The American Statistician, American Statistical Association, vol. 61, pages 262-268, August.
    2. Rajeev Dehejia, 2013. "The Porous Dialectic: Experimental and Non-Experimental Methods in Development Economics," WIDER Working Paper Series wp-2013-011, World Institute for Development Economic Research (UNU-WIDER).
    3. Tymon Słoczyński, 2015. "The Oaxaca–Blinder Unexplained Component as a Treatment Effects Estimator," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(4), pages 588-604, August.
    4. Sloczynski, Tymon, 2013. "Population Average Gender Effects," IZA Discussion Papers 7315, Institute of Labor Economics (IZA).
    5. Martin Huber, 2012. "Identification of Average Treatment Effects in Social Experiments Under Alternative Forms of Attrition," Journal of Educational and Behavioral Statistics, , vol. 37(3), pages 443-474, June.
    6. Petri Böckerman & Alex Bryson & Pekka Ilmakunnas, 2013. "Does high involvement management lead to higher pay?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(4), pages 861-885, October.
    7. Paweł Strawiński, 2013. "Controlling for overlap in matching," Working Papers 2013-10, Faculty of Economic Sciences, University of Warsaw.
    8. Giovanni Mastrobuoni & Paolo Pinotti, 2011. "Legal status of immigrants and criminal behavior: evidence from a natural experiment," Temi di discussione (Economic working papers) 813, Bank of Italy, Economic Research and International Relations Area.
    9. Villalobos Barría, Carlos & Klasen, Stephan, 2016. "The impact of SENAI's vocational training program on employment, wages, and mobility in Brazil: Lessons for Sub Saharan Africa?," The Quarterly Review of Economics and Finance, Elsevier, vol. 62(C), pages 74-96.
    10. Michele Campolieti & Morley Gunderson & Jeffrey Smith, 2014. "The effect of vocational rehabilitation on the employment outcomes of disability insurance beneficiaries: new evidence from Canada," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 3(1), pages 1-29, December.
    11. Carlos A. Flores & Oscar A. Mitnik, 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," Working Papers 2010-10, University of Miami, Department of Economics.
    12. Firpo, Sergio Pinheiro & Pinto, Rafael de Carvalho Cayres, 2012. "Combining Strategies for the Estimation of Treatment Effects," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 32(1), March.
    13. Ignacio García-Pérez, J. & Hidalgo-Hidalgo, Marisa, 2017. "No student left behind? Evidence from the Programme for School Guidance in Spain," Economics of Education Review, Elsevier, vol. 60(C), pages 97-111.
    14. Joseph G. Altonji & Prashant Bharadwaj & Fabian Lange, 2012. "Changes in the Characteristics of American Youth: Implications for Adult Outcomes," Journal of Labor Economics, University of Chicago Press, vol. 30(4), pages 783-828.
    15. Dehejia Rajeev, 2015. "Experimental and Non-Experimental Methods in Development Economics: A Porous Dialectic," Journal of Globalization and Development, De Gruyter, vol. 6(1), pages 47-69, June.
    16. A. Yalta & A. Yalta, 2010. "Should Economists Use Open Source Software for Doing Research?," Computational Economics, Springer;Society for Computational Economics, vol. 35(4), pages 371-394, April.
    17. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.
    18. Jason S. Bergtold & Krishna P. Pokharel & Allen M. Featherstone & Lijia Mo, 2018. "On the examination of the reliability of statistical software for estimating regression models with discrete dependent variables," Computational Statistics, Springer, vol. 33(2), pages 757-786, June.
    19. Ying-Ying Lee, 2015. "Efficient propensity score regression estimators of multi-valued treatment effects for the treated," Economics Series Working Papers 738, University of Oxford, Department of Economics.
    20. Tobias Brändle & Lukas Fervers, 2017. "Give it Another Try: What are the Effects of a Public Employment Scheme Especially Designed for Hard-to-Place Workers?," IAW Discussion Papers 129, Institut für Angewandte Wirtschaftsforschung (IAW).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:6278. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.