IDEAS home Printed from
   My bibliography  Save this paper

Calculating confidence intervals for continuous and discontinuous functions of parameters


  • Tiemen M. Woutersen

    (Institute for Fiscal Studies and John Hopkins University)

  • John Ham

    (Institute for Fiscal Studies and University of Maryland)


Applied researchers often need to estimate confidence intervals for functions of parameters, such as the effects of counterfactual policy changes. If the function is continuously differentiable and has non-zero and bounded derivatives, then they can use the delta method. However, if the function is nondifferentiable (as in the case of simulating functions with zero-one outcomes), has zero derivatives, or unbounded derivatives, then researchers usually use the nonparametric bootstrap or sample from the asymptotic distribution of the estimated parameter vector. Researchers also use these bootstrap approaches when the function is well-behaved but complicated. Indeed, these approaches are advocated by two very influential published articles. We first show that both of these bootstrap procedures can produce confidence intervals whose asymptotic coverage is less than advertised, i.e. confidence intervals that are too small. We then propose two procedures that provide correct coverage. In applications, we find that the bootstrap approaches mentioned above produce confidence intervals that are significantly smaller than their consistent counterparts, suggesting that previous empirical work is likely to have been overly optimistic in terms of the precision of estimated counterfactual effects.

Suggested Citation

  • Tiemen M. Woutersen & John Ham, 2013. "Calculating confidence intervals for continuous and discontinuous functions of parameters," CeMMAP working papers CWP23/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  • Handle: RePEc:ifs:cemmap:23/13

    Download full text from publisher

    File URL:
    Download Restriction: no

    References listed on IDEAS

    1. Donald W. K. Andrews, 2000. "Inconsistency of the Bootstrap when a Parameter Is on the Boundary of the Parameter Space," Econometrica, Econometric Society, vol. 68(2), pages 399-406, March.
    2. Gaure, Simen & Røed, Knut & Westlie, Lars, 2012. "Job search incentives and job match quality," Labour Economics, Elsevier, vol. 19(3), pages 438-450.
    3. Andrews, Donald W.K. & Cheng, Xu, 2013. "Maximum likelihood estimation and uniform inference with sporadic identification failure," Journal of Econometrics, Elsevier, vol. 173(1), pages 36-56.
    4. Andrew Harvey (ed.), 1994. "Time Series," Books, Edward Elgar Publishing, volume 0, number 599.
    5. Gunter J. Hitsch & Ali Hortaçsu & Dan Ariely, 2010. "Matching and Sorting in Online Dating," American Economic Review, American Economic Association, vol. 100(1), pages 130-163, March.
    6. Fitzenberger, Bernd & Osikominu, Aderonke & Paul, Marie, 2010. "The Heterogeneous Effects of Training Incidence and Duration on Labor Market Transitions," IZA Discussion Papers 5269, Institute for the Study of Labor (IZA).
    7. Donald W. K. Andrews & Xu Cheng, 2012. "Estimation and Inference With Weak, Semi‐Strong, and Strong Identification," Econometrica, Econometric Society, vol. 80(5), pages 2153-2211, September.
    8. Cameron,A. Colin & Trivedi,Pravin K., 2008. "Microeconometrics," Cambridge Books, Cambridge University Press, number 9787111235767, March.
    9. Jorg Stoye, 2009. "More on Confidence Intervals for Partially Identified Parameters," Econometrica, Econometric Society, vol. 77(4), pages 1299-1315, July.
    10. Stephen P. Ryan, 2012. "The Costs of Environmental Regulation in a Concentrated Industry," Econometrica, Econometric Society, vol. 80(3), pages 1019-1061, May.
    11. Andrews, Donald W.K. & Guggenberger, Patrik, 2010. "ASYMPTOTIC SIZE AND A PROBLEM WITH SUBSAMPLING AND WITH THE m OUT OF n BOOTSTRAP," Econometric Theory, Cambridge University Press, vol. 26(02), pages 426-468, April.
    12. Hoderlein, Stefan & Mihaleva, Sonya, 2008. "Increasing the price variation in a repeated cross section," Journal of Econometrics, Elsevier, vol. 147(2), pages 316-325, December.
    Full references (including those not matched with items on IDEAS)

    More about this item


    confidence intervals; simulation; structural models; policy effects;

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ifs:cemmap:23/13. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Emma Hyman). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.