IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/6278.html
   My bibliography  Save this paper

Perils of simulation : parallel streams and the case of stata's rnormal command

Author

Listed:
  • Ozier, Owen

Abstract

Large-scale simulation-based studies rely on at least three properties of pseudorandom number sequences: they behave in many ways like truly random numbers; they can be replicated; and they can be generated in parallel. There has been some divergence, however, between empirical techniques employing random numbers, and the standard battery of tests used to validate them. A random number generator that passes tests for any single stream of random numbers may fail the same tests when it is used to generate multiple streams in parallel. The lack of systematic testing of parallel streams leaves statistical software with important potential vulnerabilities. This paper shows one such vulnerability in Stata's rnormal function that went unnoticed for almost four years, and how to detect it. It then shows practical implications for the use of parallel streams in existing software.

Suggested Citation

  • Ozier, Owen, 2012. "Perils of simulation : parallel streams and the case of stata's rnormal command," Policy Research Working Paper Series 6278, The World Bank.
    • Handle: RePEc:wbk:wbrwps:6278
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2012/11/28/000158349_20121128134827/Rendered/PDF/wps6278.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.
    2. Nigel Smeeton & Nicholas J. Cox, 2003. "Do-it-yourself shuffling and the number of runs under randomness," Stata Journal, StataCorp LP, vol. 3(3), pages 270-277, September.
    3. B. D. McCullough, 2006. "A review of TESTU01," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(5), pages 677-682, July.
    4. Tirler, Gunter & Dalgaard, Peter & Hormann, Wolfgang & Leydold, Josef, 2004. "An error in the Kinderman-Ramage method and how to fix it," Computational Statistics & Data Analysis, Elsevier, vol. 47(3), pages 433-440, October.
    5. Busso, Matias & DiNardo, John & McCrary, Justin, 2009. "New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators," IZA Discussion Papers 3998, Institute of Labor Economics (IZA).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yalta, A. Talha, 2007. "The Numerical Reliability of GAUSS 8.0," The American Statistician, American Statistical Association, vol. 61, pages 262-268, August.
    2. Tymon Słoczyński, 2015. "The Oaxaca–Blinder Unexplained Component as a Treatment Effects Estimator," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(4), pages 588-604, August.
    3. Sloczynski, Tymon, 2013. "Population Average Gender Effects," IZA Discussion Papers 7315, Institute of Labor Economics (IZA).
    4. Petri Böckerman & Alex Bryson & Pekka Ilmakunnas, 2013. "Does high involvement management lead to higher pay?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(4), pages 861-885, October.
    5. Paweł Strawiński, 2013. "Controlling for overlap in matching," Working Papers 2013-10, Faculty of Economic Sciences, University of Warsaw.
    6. Giovanni Mastrobuoni & Paolo Pinotti, 2011. "Legal status of immigrants and criminal behavior: evidence from a natural experiment," Temi di discussione (Economic working papers) 813, Bank of Italy, Economic Research and International Relations Area.
    7. Ignacio García-Pérez, J. & Hidalgo-Hidalgo, Marisa, 2017. "No student left behind? Evidence from the Programme for School Guidance in Spain," Economics of Education Review, Elsevier, vol. 60(C), pages 97-111.
    8. Joseph G. Altonji & Prashant Bharadwaj & Fabian Lange, 2012. "Changes in the Characteristics of American Youth: Implications for Adult Outcomes," Journal of Labor Economics, University of Chicago Press, vol. 30(4), pages 783-828.
    9. Berg Claudia & Emran Shahe & Shilpi Forhad, 2020. "Microfinance and Moneylenders: Long-run Effects of MFIs on Informal Credit Market in Bangladesh," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 20(3), pages 1-35, July.
    10. Giovanni Mastrobuoni & Paolo Pinotti, 2015. "Legal Status and the Criminal Activity of Immigrants," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 175-206, April.
    11. Mastrobuoni, Giovanni & Pinotti, Paolo, 2011. "Migration Restrictions and Criminal Behavior: Evidence from a Natural Experiment," Economy and Society 115723, Fondazione Eni Enrico Mattei (FEEM).
    12. Maria Guadalupe & Olga Kuzmina & Catherine Thomas, 2012. "Innovation and Foreign Ownership," American Economic Review, American Economic Association, vol. 102(7), pages 3594-3627, December.
    13. Andersson, Fredrik W. & Holzer, Harry J. & Lane, Julia & Rosenblum, David & Smith, Jeffrey A., 2013. "Does Federally-Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," IZA Discussion Papers 7621, Institute of Labor Economics (IZA).
    14. Meredith Fowlie & Stephen P. Holland & Erin T. Mansur, 2012. "What Do Emissions Markets Deliver and to Whom? Evidence from Southern California's NOx Trading Program," American Economic Review, American Economic Association, vol. 102(2), pages 965-993, April.
    15. Annabelle Doerr & Bernd Fitzenberger & Thomas Kruppe & Marie Paul & Anthony Strittmatter, 2017. "Employment and Earnings Effects of Awarding Training Vouchers in Germany," ILR Review, Cornell University, ILR School, vol. 70(3), pages 767-812, May.
    16. Muller, Paul & van der Klaauw, Bas & Heyma, Arjan, 2017. "Comparing Econometric Methods to Empirically Evaluate Job-Search Assistance," IZA Discussion Papers 10531, Institute of Labor Economics (IZA).
    17. Goerke, Laszlo & Pannenberg, Markus, 2015. "Trade union membership and sickness absence: Evidence from a sick pay reform," Labour Economics, Elsevier, vol. 33(C), pages 13-25.
    18. Caliendo, Marco & Künn, Steffen & Schmidl, Ricarda, 2011. "Fighting Youth Unemployment: The Effects of Active Labor Market Policies," IZA Discussion Papers 6222, Institute of Labor Economics (IZA).
    19. Martin Huber & Michael Lechner & Andreas Steinmayr, 2015. "Radius matching on the propensity score with bias adjustment: tuning parameters and finite sample behaviour," Empirical Economics, Springer, vol. 49(1), pages 1-31, August.
    20. Steven Lehrer & Gregory Kordas, 2013. "Matching using semiparametric propensity scores," Empirical Economics, Springer, vol. 44(1), pages 13-45, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:6278. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.