IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/6278.html

Perils of simulation : parallel streams and the case of stata's rnormal command

Author

Listed:
  • Ozier, Owen

Abstract

Large-scale simulation-based studies rely on at least three properties of pseudorandom number sequences: they behave in many ways like truly random numbers; they can be replicated; and they can be generated in parallel. There has been some divergence, however, between empirical techniques employing random numbers, and the standard battery of tests used to validate them. A random number generator that passes tests for any single stream of random numbers may fail the same tests when it is used to generate multiple streams in parallel. The lack of systematic testing of parallel streams leaves statistical software with important potential vulnerabilities. This paper shows one such vulnerability in Stata's rnormal function that went unnoticed for almost four years, and how to detect it. It then shows practical implications for the use of parallel streams in existing software.

Suggested Citation

  • Ozier, Owen, 2012. "Perils of simulation : parallel streams and the case of stata's rnormal command," Policy Research Working Paper Series 6278, The World Bank.
    • Handle: RePEc:wbk:wbrwps:6278
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2012/11/28/000158349_20121128134827/Rendered/PDF/wps6278.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Busso, Matias & DiNardo, John & McCrary, Justin, 2009. "New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators," IZA Discussion Papers 3998, Institute of Labor Economics (IZA).
    2. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.
    3. Nigel Smeeton & Nicholas J. Cox, 2003. "Do-it-yourself shuffling and the number of runs under randomness," Stata Journal, StataCorp LLC, vol. 3(3), pages 270-277, September.
    4. B. D. McCullough, 2006. "A review of TESTU01," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(5), pages 677-682, July.
    5. Tirler, Gunter & Dalgaard, Peter & Hormann, Wolfgang & Leydold, Josef, 2004. "An error in the Kinderman-Ramage method and how to fix it," Computational Statistics & Data Analysis, Elsevier, vol. 47(3), pages 433-440, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yalta, A. Talha, 2007. "The Numerical Reliability of GAUSS 8.0," The American Statistician, American Statistical Association, vol. 61, pages 262-268, August.
    2. Sloczynski, Tymon, 2013. "Population Average Gender Effects," IZA Discussion Papers 7315, Institute of Labor Economics (IZA).
    3. Paweł Strawiński, 2013. "Controlling for overlap in matching," Working Papers 2013-10, Faculty of Economic Sciences, University of Warsaw.
    4. Giovanni Mastrobuoni & Paolo Pinotti, 2011. "Legal status of immigrants and criminal behavior: evidence from a natural experiment," Temi di discussione (Economic working papers) 813, Bank of Italy, Economic Research and International Relations Area.
    5. Joseph G. Altonji & Prashant Bharadwaj & Fabian Lange, 2012. "Changes in the Characteristics of American Youth: Implications for Adult Outcomes," Journal of Labor Economics, University of Chicago Press, vol. 30(4), pages 783-828.
    6. Giovanni Mastrobuoni & Paolo Pinotti, 2015. "Legal Status and the Criminal Activity of Immigrants," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 175-206, April.
    7. Mastrobuoni, Giovanni & Pinotti, Paolo, 2011. "Migration Restrictions and Criminal Behavior: Evidence from a Natural Experiment," Economy and Society 115723, Fondazione Eni Enrico Mattei (FEEM).
    8. Meredith Fowlie & Stephen P. Holland & Erin T. Mansur, 2012. "What Do Emissions Markets Deliver and to Whom? Evidence from Southern California's NOx Trading Program," American Economic Review, American Economic Association, vol. 102(2), pages 965-993, April.
    9. Annabelle Doerr & Bernd Fitzenberger & Thomas Kruppe & Marie Paul & Anthony Strittmatter, 2017. "Employment and Earnings Effects of Awarding Training Vouchers in Germany," ILR Review, Cornell University, ILR School, vol. 70(3), pages 767-812, May.
    10. Muller, Paul & van der Klaauw, Bas & Heyma, Arjan, 2017. "Comparing Econometric Methods to Empirically Evaluate Job-Search Assistance," IZA Discussion Papers 10531, Institute of Labor Economics (IZA).
    11. Goerke, Laszlo & Pannenberg, Markus, 2015. "Trade union membership and sickness absence: Evidence from a sick pay reform," Labour Economics, Elsevier, vol. 33(C), pages 13-25.
    12. Caliendo, Marco & Künn, Steffen & Schmidl, Ricarda, 2011. "Fighting Youth Unemployment: The Effects of Active Labor Market Policies," IZA Discussion Papers 6222, Institute of Labor Economics (IZA).
    13. Martin Huber & Michael Lechner & Andreas Steinmayr, 2015. "Radius matching on the propensity score with bias adjustment: tuning parameters and finite sample behaviour," Empirical Economics, Springer, vol. 49(1), pages 1-31, August.
    14. Kevin Stange, 2011. "A Longitudinal Analysis of the Relationship Between Fertility Timing and Schooling," Demography, Springer;Population Association of America (PAA), vol. 48(3), pages 931-956, August.
    15. Holger Görg & Erasmus Kersting, 2017. "Vertical integration and supplier finance," Canadian Journal of Economics, Canadian Economics Association, vol. 50(1), pages 273-305, February.
    16. Hugo Bodory & Lorenzo Camponovo & Martin Huber & Michael Lechner, 2020. "The Finite Sample Performance of Inference Methods for Propensity Score Matching and Weighting Estimators," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(1), pages 183-200, January.
    17. Fortin, Nicole & Lemieux, Thomas & Firpo, Sergio, 2011. "Decomposition Methods in Economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 1, pages 1-102, Elsevier.
    18. James Levinsohn & Zoë M. McLaren & Olive Shisana & Khangelani Zuma, 2013. "HIV Status and Labor Market Participation in South Africa," The Review of Economics and Statistics, MIT Press, vol. 95(1), pages 98-108, March.
    19. Samaniego de la Parra Brenda & Fernández Bujanda León, 2020. "Increasing the Cost of Informal Workers: Evidence from Mexico," Working Papers 2020-19, Banco de México.
    20. Gustavo Canavire-Bacarreza & Luis Castro Peñarrieta & Darwin Ugarte Ontiveros, 2021. "Outliers in Semi-Parametric Estimation of Treatment Effects," Econometrics, MDPI, vol. 9(2), pages 1-32, April.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:6278. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.