IDEAS home Printed from https://ideas.repec.org/a/tsj/stataj/v19y2019i1p4-60.html
   My bibliography  Save this article

Fast and wild: Bootstrap inference in Stata using boottest

Author

Listed:
  • David Roodman

    () (Open Philanthropy Project)

  • James G. MacKinnon

    () (Queen’s University)

  • Morten Ørregaard Nielsen

    () (Queen’s University)

  • Matthew D. Webb

    () (Carleton University)

Abstract

The wild bootstrap was originally developed for regression models with heteroskedasticity of unknown form. Over the past 30 years, it has been extended to models estimated by instrumental variables and maximum likelihood and to ones where the error terms are (perhaps multiway) clustered. Like boot- strap methods in general, the wild bootstrap is especially useful when conventional inference methods are unreliable because large-sample assumptions do not hold. For example, there may be few clusters, few treated clusters, or weak instruments. The package boottest can perform a wide variety of wild bootstrap tests, often at remarkable speed. It can also invert these tests to construct confidence sets. As a postestimation command, boottest works after linear estimation commands, in- cluding regress, cnsreg, ivregress, ivreg2, areg, and reghdfe, as well as many estimation commands based on maximum likelihood. Although it is designed to perform the wild cluster bootstrap, boottest can also perform the ordinary (non- clustered) version. Wrappers offer classical Wald, score/Lagrange multiplier, and Anderson–Rubin tests, optionally with (multiway) clustering. We review the main ideas of the wild cluster bootstrap, offer tips for use, explain why it is particularly amenable to computational optimization, state the syntax of boottest, artest, scoretest, and waldtest, and present several empirical examples.

Suggested Citation

  • David Roodman & James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2019. "Fast and wild: Bootstrap inference in Stata using boottest," Stata Journal, StataCorp LP, vol. 19(1), pages 4-60, March.
  • Handle: RePEc:tsj:stataj:v:19:y:2019:i:1:p:4-60
    DOI: 10.1177/1536867X19830877
    Note: to access software from within Stata, net describe http://www.stata-journal.com/software/sj19-1/st0549/
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1177/1536867X19830877
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. James H. Stock & Mark W. Watson, 2008. "Heteroskedasticity-Robust Standard Errors for Fixed Effects Panel Data Regression," Econometrica, Econometric Society, vol. 76(1), pages 155-174, January.
    2. repec:clg:wpaper:2013-20 is not listed on IDEAS
    3. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    4. Davidson, Russell & Flachaire, Emmanuel, 2008. "The wild bootstrap, tamed at last," Journal of Econometrics, Elsevier, vol. 146(1), pages 162-169, September.
    5. Keith Finlay & Leandro Magnusson & Mark E Schaffer, 2013. "WEAKIV: Stata module to perform weak-instrument-robust tests and confidence intervals for instrumental-variable (IV) estimation of linear, probit and tobit models," Statistical Software Components S457684, Boston College Department of Economics, revised 18 Oct 2016.
    6. Timothy G. Conley & Christopher R. Taber, 2011. "Inference with "Difference in Differences" with a Small Number of Policy Changes," The Review of Economics and Statistics, MIT Press, vol. 93(1), pages 113-125, February.
    7. Djogbenou, Antoine A. & MacKinnon, James G. & Nielsen, Morten Ørregaard, 2019. "Asymptotic theory and wild bootstrap inference with clustered errors," Journal of Econometrics, Elsevier, vol. 212(2), pages 393-412.
    8. Thompson, Samuel B., 2011. "Simple formulas for standard errors that cluster by both firm and time," Journal of Financial Economics, Elsevier, vol. 99(1), pages 1-10, January.
    9. David Roodman, 2011. "Fitting fully observed recursive mixed-process models with cmp," Stata Journal, StataCorp LP, vol. 11(2), pages 159-206, June.
    10. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    11. Davidson, Russell & MacKinnon, James G., 1993. "Estimation and Inference in Econometrics," OUP Catalogue, Oxford University Press, number 9780195060119.
    12. Steven D. Levitt, 1996. "The Effect of Prison Population Size on Crime Rates: Evidence from Prison Overcrowding Litigation," The Quarterly Journal of Economics, Oxford University Press, vol. 111(2), pages 319-351.
    13. Christopher F Baum & Mark E. Schaffer & Steven Stillman, 2007. "Enhanced routines for instrumental variables/GMM estimation and testing," Boston College Working Papers in Economics 667, Boston College Department of Economics, revised 05 Sep 2007.
    14. Davidson, James & Monticini, Andrea & Peel, David, 2007. "Implementing the wild bootstrap using a two-point distribution," Economics Letters, Elsevier, vol. 96(3), pages 309-315, September.
    15. Matthew D. Webb, 2014. "Reworking Wild Bootstrap Based Inference For Clustered Errors," Working Paper 1315, Economics Department, Queen's University.
    16. Kline Patrick & Santos Andres, 2012. "A Score Based Approach to Wild Bootstrap Inference," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 23-41, August.
    17. Jean-Marie Dufour, 1997. "Some Impossibility Theorems in Econometrics with Applications to Structural and Dynamic Models," Econometrica, Econometric Society, vol. 65(6), pages 1365-1388, November.
    18. Laurent Davezies & Xavier D'Haultfoeuille & Yannick Guyonvarch, 2018. "Asymptotic results under multiway clustering," Papers 1807.07925, arXiv.org, revised Aug 2018.
    19. Stelios Michalopoulos & Elias Papaioannou, 2013. "Pre‐Colonial Ethnic Institutions and Contemporary African Development," Econometrica, Econometric Society, vol. 81(1), pages 113-152, January.
    20. Andrew V. Carter & Kevin T. Schnepel & Douglas G. Steigerwald, 2017. "Asymptotic Behavior of a t -Test Robust to Cluster Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 99(4), pages 698-709, July.
    21. Keith Finlay & Leandro M. Magnusson, 2014. "Bootstrap Methods for Inference with Cluster-Sample IV Models," Economics Discussion / Working Papers 14-12, The University of Western Australia, Department of Economics.
    22. Bester, C. Alan & Conley, Timothy G. & Hansen, Christian B., 2011. "Inference with dependent data using cluster covariance estimators," Journal of Econometrics, Elsevier, vol. 165(2), pages 137-151.
    23. William Gould, 2010. "Mata Matters: Stata in Mata," Stata Journal, StataCorp LP, vol. 10(1), pages 125-142, March.
    24. Hansen, Christian B., 2007. "Asymptotic properties of a robust variance matrix estimator for panel data when T is large," Journal of Econometrics, Elsevier, vol. 141(2), pages 597-620, December.
    25. Jonathan Gruber & James M. Poterba, 1993. "Tax Incentives and the Decision to Purchase Health Insurance: Evidence from the Self-Employed," NBER Working Papers 4435, National Bureau of Economic Research, Inc.
    26. James G. MacKinnon & Matthew D. Webb, 2017. "Pitfalls When Estimating Treatment Effects Using Clustered Data," Working Paper 1387, Economics Department, Queen's University.
    27. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, Oxford University Press, vol. 119(1), pages 249-275.
    28. Jonathan Gruber & James Poterba, 1994. "Tax Incentives and the Decision to Purchase Health Insurance: Evidence from the Self-Employed," The Quarterly Journal of Economics, Oxford University Press, vol. 109(3), pages 701-733.
    29. C. A. Field & A. H. Welsh, 2007. "Bootstrapping clustered data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(3), pages 369-390, June.
    30. David Roodman & Jonathan Morduch, 2014. "The Impact of Microcredit on the Poor in Bangladesh: Revisiting the Evidence," Journal of Development Studies, Taylor & Francis Journals, vol. 50(4), pages 583-604, April.
    31. James G. MacKinnon & Matthew D. Webb & Morten Ø. Nielsen, 2017. "Bootstrap And Asymptotic Inference With Multiway Clustering," Working Paper 1386, Economics Department, Queen's University.
    32. Mark M. Pitt & Shahidur R. Khandker, 1998. "The Impact of Group-Based Credit Programs on Poor Households in Bangladesh: Does the Gender of Participants Matter?," Journal of Political Economy, University of Chicago Press, vol. 106(5), pages 958-996, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. James G. MacKinnon & Matthew D. Webb, 2020. "When and How to Deal with Clustered Errors in Regression Models," Working Paper 1421, Economics Department, Queen's University.
    2. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    3. Djogbenou, Antoine A. & MacKinnon, James G. & Nielsen, Morten Ørregaard, 2019. "Asymptotic theory and wild bootstrap inference with clustered errors," Journal of Econometrics, Elsevier, vol. 212(2), pages 393-412.
    4. Ivan A. Canay & Andres Santos & Azeem M. Shaikh, 2018. "The wild bootstrap with a "small" number of "large" clusters," CeMMAP working papers CWP27/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    5. James G. MacKinnon & Morten Ø. Nielsen & Matthew D. Webb, 2019. "Wild Bootstrap and Asymptotic Inference with Multiway Clustering," Working Paper 1415, Economics Department, Queen's University.
    6. James G. MacKinnon, 2019. "How cluster-robust inference is changing applied econometrics," Canadian Journal of Economics, Canadian Economics Association, vol. 52(3), pages 851-881, August.
    7. James G. MacKinnon & Matthew D. Webb, 2017. "Pitfalls When Estimating Treatment Effects Using Clustered Data," Working Paper 1387, Economics Department, Queen's University.
    8. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2020. "Testing for the appropriate level of clustering in linear regression models," Working Paper 1428, Economics Department, Queen's University.
    9. Matthew D. Webb, 2014. "Reworking Wild Bootstrap Based Inference For Clustered Errors," Working Paper 1315, Economics Department, Queen's University.
    10. James G. MacKinnon & Matthew D. Webb & Morten Ø. Nielsen, 2017. "Bootstrap And Asymptotic Inference With Multiway Clustering," Working Paper 1386, Economics Department, Queen's University.
    11. Hagemann, Andreas, 2019. "Placebo inference on treatment effects when the number of clusters is small," Journal of Econometrics, Elsevier, vol. 213(1), pages 190-209.
    12. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 106, University of California, Davis, Department of Economics.
    13. Bruno Ferman, 2019. "Inference in Differences-in-Differences: How Much Should We Trust in Independent Clusters?," Papers 1909.01782, arXiv.org, revised Sep 2020.
    14. Antoine A. Djogbenou & James G. MacKinnon & Morten Ø. Nielsen, 2017. "Validity Of Wild Bootstrap Inference With Clustered Errors," Working Paper 1383, Economics Department, Queen's University.
    15. Andreas Hagemann, 2019. "Permutation inference with a finite number of heterogeneous clusters," Papers 1907.01049, arXiv.org.
    16. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 318, University of California, Davis, Department of Economics.
    17. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    18. James G. MacKinnon & Matthew D. Webb, 2017. "Wild Bootstrap Inference for Wildly Different Cluster Sizes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(2), pages 233-254, March.
    19. Packalen, Mikko & Wirjanto, Tony S., 2012. "Inference about clustering and parametric assumptions in covariance matrix estimation," Computational Statistics & Data Analysis, Elsevier, vol. 56(1), pages 1-14, January.
    20. repec:fgv:eesptd:411 is not listed on IDEAS
    21. Timothy Conley & Silvia Gonçalves & Christian Hansen, 2018. "Inference with Dependent Data in Accounting and Finance Applications," Journal of Accounting Research, Wiley Blackwell, vol. 56(4), pages 1139-1203, September.

    More about this item

    Keywords

    boottest; artest; waldtest; scoretest; Anderson–Rubin test; Wald test; wild bootstrap; wild cluster bootstrap; score bootstrap; multiway clustering; few treated clusters;
    All these keywords.

    JEL classification:

    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • C36 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Instrumental Variables (IV) Estimation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tsj:stataj:v:19:y:2019:i:1:p:4-60. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Christopher F. Baum) or (Lisa Gilmore). General contact details of provider: http://www.stata-journal.com/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.