IDEAS home Printed from https://ideas.repec.org/p/qed/wpaper/995.html

Improving The Reliability Of Bootstrap Tests

Author

Listed:
  • James G. MacKinnon

    (Queen's University)

  • Russell Davidson

    (McGill University)

Abstract

We first propose procedures for estimating the rejection probabilities for bootstrap tests in Monte Carlo experiments without actually computing a bootstrap test for each replication. These procedures are only about twice as expensive as estimating rejection probabilities for asymptotic tersts. We then propose procedures for computing modified bootstrap P values that will often be more accurate than ordinary ones. These procedures are closely related to the double bootstrap, but they are far less computationally demanding.

Suggested Citation

  • James G. MacKinnon & Russell Davidson, 2000. "Improving The Reliability Of Bootstrap Tests," Working Paper 995, Economics Department, Queen's University.
    • Handle: RePEc:qed:wpaper:995
    as

    Download full text from publisher

    File URL: https://www.econ.queensu.ca/sites/econ.queensu.ca/files/qed_wp_995.pdf
    File Function: First version 2000
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Horowitz, Joel L., 1994. "Bootstrap-based critical values for the information matrix test," Journal of Econometrics, Elsevier, vol. 61(2), pages 395-411, April.
    2. Davidson, Russell & MacKinnon, James G., 1999. "The Size Distortion Of Bootstrap Tests," Econometric Theory, Cambridge University Press, vol. 15(3), pages 361-376, June.
    3. Davidson, Russell & MacKinnon, James G, 1999. "Bootstrap Testing in Nonlinear Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 40(2), pages 487-508, May.
    4. Davidson, Russell & MacKinnon, James G, 1998. "Graphical Methods for Investigating the Size and Power of Hypothesis Tests," The Manchester School of Economic & Social Studies, University of Manchester, vol. 66(1), pages 1-26, January.
    5. Davidson, Russell & MacKinnon, James G., 1984. "Convenient specification tests for logit and probit models," Journal of Econometrics, Elsevier, vol. 25(3), pages 241-262, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jaap W. B. Bos & Bertrand Candelon & Claire Economidou, 2016. "Does knowledge spill over across borders and technology regimes?," Journal of Productivity Analysis, Springer, vol. 46(1), pages 63-82, August.
    2. Bruggeman, Annick & Donati, Paola & Warne, Anders, 2003. "Is the demand for euro area M3 stable?," Working Paper Series 255, European Central Bank.
    3. repec:use:tkiwps:3232 is not listed on IDEAS
    4. Omtzigt Pieter & Fachin Stefano, 2002. "Bootstrapping and Bartlett corrections in the cointegrated VAR model," Economics and Quantitative Methods qf0212, Department of Economics, University of Insubria.
    5. Timothy Patrick Moran, 2006. "Statistical Inference for Measures of Inequality With a Cross-National Bootstrap Application," Sociological Methods & Research, , vol. 34(3), pages 296-333, February.
    6. Davidson, Russell & MacKinnon, James G., 2007. "Improving the reliability of bootstrap tests with the fast double bootstrap," Computational Statistics & Data Analysis, Elsevier, vol. 51(7), pages 3259-3281, April.
    7. Bodart, V. & Candelon, B. & Carpantier, J.-F., 2012. "Real exchanges rates in commodity producing countries: A reappraisal," Journal of International Money and Finance, Elsevier, vol. 31(6), pages 1482-1502.
    8. Brücker, Herbert & Fachin, Stefano & Venturini, Alessandra, 2011. "Do foreigners replace native immigrants? A panel cointegration analysis of internal migration in Italy," Economic Modelling, Elsevier, vol. 28(3), pages 1078-1089, May.
    9. Psaradakis, Zacharias, 2006. "Blockwise bootstrap testing for stationarity," Statistics & Probability Letters, Elsevier, vol. 76(6), pages 562-570, March.
    10. James Morley & Irina B. Panovska & Tara M. Sinclair, 2014. "Testing Stationarity for Unobserved Components Models," Discussion Papers 2012-41B, School of Economics, The University of New South Wales.
    11. Pieter Omtzigt & Stefano Fachin, 2006. "The Size and Power of Bootstrap and Bartlett-Corrected Tests of Hypotheses on the Cointegrating Vectors," Econometric Reviews, Taylor & Francis Journals, vol. 25(1), pages 41-60.
    12. Giacomini, Raffaella & Politis, Dimitris N. & White, Halbert, 2013. "A Warp-Speed Method For Conducting Monte Carlo Experiments Involving Bootstrap Estimators," Econometric Theory, Cambridge University Press, vol. 29(3), pages 567-589, June.
    13. Stefano Fachin, 2007. "Long-run trends in internal migrations in italy: a study in panel cointegration with dependent units," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(2), pages 401-428.
    14. Basher, Syed A. & Fachin, Stefano, 2008. "The long-term decline of internal migration in Canada – Ontario as a case study," MPRA Paper 6685, University Library of Munich, Germany.
    15. Grigory Franguridi & Lidia Kosenkova, 2024. "Closed-form estimation and inference for panels with attrition and refreshment samples," Papers 2410.11263, arXiv.org, revised Jul 2025.
    16. Stefano Fachin & Andrea Gavosto, 2010. "Trends of labour productivity in Italy: a study with panel co‐integration methods," International Journal of Manpower, Emerald Group Publishing Limited, vol. 31(7), pages 755-769, October.
    17. Matteo Mogliani & Giovanni Urga & Carlos Winograd, 2009. "Monetary disorder and financial regimes - The demand for money in Argentina, 1900-2006," PSE Working Papers halshs-00575107, HAL.
    18. Russell Davidson & James MacKinnon, 2002. "Fast Double Bootstrap Tests Of Nonnested Linear Regression Models," Econometric Reviews, Taylor & Francis Journals, vol. 21(4), pages 419-429.
    19. Davidson, James, 2006. "Alternative bootstrap procedures for testing cointegration in fractionally integrated processes," Journal of Econometrics, Elsevier, vol. 133(2), pages 741-777, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Davidson, Russell & MacKinnon, James G., 2006. "The power of bootstrap and asymptotic tests," Journal of Econometrics, Elsevier, vol. 133(2), pages 421-441, August.
    2. Riccardo Lucchetti & Claudia Pigini, 2013. "A test for bivariate normality with applications in microeconometric models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 22(4), pages 535-572, November.
    3. Davidson, Russell & MacKinnon, James G., 1996. "The Power of Bootstrap Tests," Queen's Institute for Economic Research Discussion Papers 273372, Queen's University - Department of Economics.
    4. Davidson, Russell & MacKinnon, James G., 2002. "Bootstrap J tests of nonnested linear regression models," Journal of Econometrics, Elsevier, vol. 109(1), pages 167-193, July.
    5. Russell Davidson & James MacKinnon, 2000. "Bootstrap tests: how many bootstraps?," Econometric Reviews, Taylor & Francis Journals, vol. 19(1), pages 55-68.
    6. Davidson, Russell & MacKinnon, James G., 2007. "Improving the reliability of bootstrap tests with the fast double bootstrap," Computational Statistics & Data Analysis, Elsevier, vol. 51(7), pages 3259-3281, April.
    7. de Peretti, Christian & Siani, Carole, 2010. "Graphical methods for investigating the finite-sample properties of confidence regions," Computational Statistics & Data Analysis, Elsevier, vol. 54(2), pages 262-271, February.
    8. Davidson, Russell & MacKinnon, James G., 1997. "Bootstrap Testing How Many Bootstraps," Queen's Institute for Economic Research Discussion Papers 273385, Queen's University - Department of Economics.
    9. Christian de Peretti, 2003. "Bilateral Bootstrap Tests for Long Memory: An Application to the Silver Market," Computational Economics, Springer;Society for Computational Economics, vol. 22(2), pages 187-212, October.
    10. Jamel Jouini, 2006. "Bootstrap Tests in Bivariate VAR Process with Single Structural Change : Power versus Corrected Size and Empirical Illustration," Working Papers halshs-00410759, HAL.
    11. Dirk Hoorelbeke, 2004. "Bootstrap correcting the score test," Econometric Society 2004 North American Summer Meetings 228, Econometric Society.
    12. Mantalos Panagiotis, 2003. "Bootstrapping the Breusch-Godfrey autocorrelation test for a single equation dynamic model: Bootstrapping the Restricted vs. Unrestricted model," Monte Carlo Methods and Applications, De Gruyter, vol. 9(3), pages 257-269, September.
    13. Timothy Patrick Moran, 2006. "Statistical Inference for Measures of Inequality With a Cross-National Bootstrap Application," Sociological Methods & Research, , vol. 34(3), pages 296-333, February.
    14. Davidson, Russell & Flachaire, Emmanuel, 2007. "Asymptotic and bootstrap inference for inequality and poverty measures," Journal of Econometrics, Elsevier, vol. 141(1), pages 141-166, November.
    15. Christian Peretti, 2007. "Long Memory and Hysteresis," Springer Books, in: Gilles Teyssière & Alan P. Kirman (ed.), Long Memory in Economics, pages 363-389, Springer.
    16. Dufour, Jean-Marie & Khalaf, Lynda, 2002. "Simulation based finite and large sample tests in multivariate regressions," Journal of Econometrics, Elsevier, vol. 111(2), pages 303-322, December.
    17. Bravo, Francesco & Crudu, Federico, 2012. "Efficient bootstrap with weakly dependent processes," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3444-3458.
    18. Chiu, Sung Nok & Wang, Ling, 2009. "Homogeneity tests for several Poisson populations," Computational Statistics & Data Analysis, Elsevier, vol. 53(12), pages 4266-4278, October.
    19. Kasahara, Hiroyuki & Shimotsu, Katsumi, 2008. "Pseudo-likelihood estimation and bootstrap inference for structural discrete Markov decision models," Journal of Econometrics, Elsevier, vol. 146(1), pages 92-106, September.
    20. Flachaire, Emmanuel, 1999. "A better way to bootstrap pairs," Economics Letters, Elsevier, vol. 64(3), pages 257-262, September.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qed:wpaper:995. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Babcock (email available below). General contact details of provider: https://edirc.repec.org/data/qedquca.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.