IDEAS home Printed from https://ideas.repec.org/p/ifs/cemmap/23-13.html
   My bibliography  Save this paper

Calculating confidence intervals for continuous and discontinuous functions of parameters

Author

Listed:
  • Tiemen M. Woutersen

    (Institute for Fiscal Studies and John Hopkins University)

  • John Ham

    (Institute for Fiscal Studies and University of Maryland)

Abstract

Applied researchers often need to estimate confidence intervals for functions of parameters, such as the effects of counterfactual policy changes. If the function is continuously differentiable and has non-zero and bounded derivatives, then they can use the delta method. However, if the function is nondifferentiable (as in the case of simulating functions with zero-one outcomes), has zero derivatives, or unbounded derivatives, then researchers usually use the nonparametric bootstrap or sample from the asymptotic distribution of the estimated parameter vector. Researchers also use these bootstrap approaches when the function is well-behaved but complicated. Indeed, these approaches are advocated by two very influential published articles. We first show that both of these bootstrap procedures can produce confidence intervals whose asymptotic coverage is less than advertised, i.e. confidence intervals that are too small. We then propose two procedures that provide correct coverage. In applications, we find that the bootstrap approaches mentioned above produce confidence intervals that are significantly smaller than their consistent counterparts, suggesting that previous empirical work is likely to have been overly optimistic in terms of the precision of estimated counterfactual effects.

Suggested Citation

  • Tiemen M. Woutersen & John Ham, 2013. "Calculating confidence intervals for continuous and discontinuous functions of parameters," CeMMAP working papers CWP23/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  • Handle: RePEc:ifs:cemmap:23/13
    as

    Download full text from publisher

    File URL: http://www.cemmap.ac.uk/wps/cwp231313.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Runkle, David E, 1987. "Vector Autoregressions and Reality," Journal of Business & Economic Statistics, American Statistical Association, vol. 5(4), pages 437-442, October.
    2. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    3. Donald W. K. Andrews, 2000. "Inconsistency of the Bootstrap when a Parameter Is on the Boundary of the Parameter Space," Econometrica, Econometric Society, vol. 68(2), pages 399-406, March.
    4. Keane, Michael P & Wolpin, Kenneth I, 2000. "Eliminating Race Differences in School Attainment and Labor Market Success," Journal of Labor Economics, University of Chicago Press, vol. 18(4), pages 614-652, October.
    5. Inoue, Atsushi & Kilian, Lutz, 2013. "Inference on impulse response functions in structural VAR models," Journal of Econometrics, Elsevier, vol. 177(1), pages 1-13.
    6. Panle Jia & Pinelopi K. Goldberg & Shubham Chaudhuri, 2006. "Estimating the Effects of Global Patent Protection in Pharmaceuticals: A Case Study of Quinolones in India," American Economic Review, American Economic Association, vol. 96(5), pages 1477-1514, December.
    7. Gaure, Simen & Røed, Knut & Westlie, Lars, 2012. "Job search incentives and job match quality," Labour Economics, Elsevier, vol. 19(3), pages 438-450.
    8. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    9. Andrews, Donald W K, 1987. "Consistency in Nonlinear Econometric Models: A Generic Uniform Law of Large Numbers [On Unification of the Asymptotic Theory of Nonlinear Econometric Models]," Econometrica, Econometric Society, vol. 55(6), pages 1465-1471, November.
    10. Andrews, Donald W.K. & Cheng, Xu, 2013. "Maximum likelihood estimation and uniform inference with sporadic identification failure," Journal of Econometrics, Elsevier, vol. 173(1), pages 36-56.
    11. Andrew Harvey (ed.), 1994. "Time Series," Books, Edward Elgar Publishing, volume 0, number 599.
    12. Gunter J. Hitsch & Ali Hortaçsu & Dan Ariely, 2010. "Matching and Sorting in Online Dating," American Economic Review, American Economic Association, vol. 100(1), pages 130-163, March.
    13. Antonio Merlo & Kenneth I. Wolpin, 2008. "The Transition from School to Jail: Youth Crime and High School Completion Among Black Males, Second Version," PIER Working Paper Archive 09-002, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 16 Jan 2009.
    14. Fitzenberger, Bernd & Osikominu, Aderonke & Paul, Marie, 2010. "The Heterogeneous Effects of Training Incidence and Duration on Labor Market Transitions," IZA Discussion Papers 5269, Institute of Labor Economics (IZA).
    15. Donald W. K. Andrews & Xu Cheng, 2012. "Estimation and Inference With Weak, Semi‐Strong, and Strong Identification," Econometrica, Econometric Society, vol. 80(5), pages 2153-2211, September.
    16. Cameron,A. Colin & Trivedi,Pravin K., 2005. "Microeconometrics," Cambridge Books, Cambridge University Press, number 9780521848053.
    17. Curtis Eberwein & John C. Ham & Robert J. Lalonde, 1997. "The Impact of Being Offered and Receiving Classroom Training on the Employment Histories of Disadvantaged Women: Evidence from Experimental Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 655-682.
    18. A. Colin Cameron & Pravin K. Trivedi, 2010. "Microeconometrics Using Stata, Revised Edition," Stata Press books, StataCorp LP, number musr, March.
    19. Eberwein, Curtis & Ham, John C. & LaLonde, Robert J., 2002. "Alternative methods of estimating program effects in event history models," Labour Economics, Elsevier, vol. 9(2), pages 249-278, April.
    20. David E. Runkle, 1987. "Vector autoregressions and reality," Staff Report 107, Federal Reserve Bank of Minneapolis.
    21. Jorg Stoye, 2009. "More on Confidence Intervals for Partially Identified Parameters," Econometrica, Econometric Society, vol. 77(4), pages 1299-1315, July.
    22. Stephen P. Ryan, 2012. "The Costs of Environmental Regulation in a Concentrated Industry," Econometrica, Econometric Society, vol. 80(3), pages 1019-1061, May.
    23. Andrews, Donald W.K. & Guggenberger, Patrik, 2010. "ASYMPTOTIC SIZE AND A PROBLEM WITH SUBSAMPLING AND WITH THE m OUT OF n BOOTSTRAP," Econometric Theory, Cambridge University Press, vol. 26(2), pages 426-468, April.
    24. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    25. Krinsky, Itzhak & Robb, A Leslie, 1986. "On Approximating the Statistical Properties of Elasticities," The Review of Economics and Statistics, MIT Press, vol. 68(4), pages 715-719, November.
    26. Hoderlein, Stefan & Mihaleva, Sonya, 2008. "Increasing the price variation in a repeated cross section," Journal of Econometrics, Elsevier, vol. 147(2), pages 316-325, December.
    27. Zvi Eckstein & Kenneth I. Wolpin, 1999. "Why Youths Drop Out of High School: The Impact of Preferences, Opportunities, and Abilities," Econometrica, Econometric Society, vol. 67(6), pages 1295-1340, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Valérie Lechene & Krishna Pendakur & Alexander Wolf, 2020. "OLS estimation of the intra-household distribution of expenditure," IFS Working Papers W20/6, Institute for Fiscal Studies.
    2. Lee, Ying-Ying & Bhattacharya, Debopam, 2019. "Applied welfare analysis for discrete choice with interval-data on income," Journal of Econometrics, Elsevier, vol. 211(2), pages 361-387.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ham, John C. & Woutersen, Tiemen, 2011. "Calculating Confidence Intervals for Continuous and Discontinuous Functions of Estimated Parameters," IZA Discussion Papers 5816, Institute of Labor Economics (IZA).
    2. Jui-Chung Yang & Ke-Li Xu, 2013. "Estimation and Inference under Weak Identi cation and Persistence: An Application on Forecast-Based Monetary Policy Reaction Function," 2013 Papers pya307, Job Market Papers.
    3. Woutersen, Tiemen & Hausman, Jerry A., 2019. "Increasing the power of specification tests," Journal of Econometrics, Elsevier, vol. 211(1), pages 166-175.
    4. Jeffrey M. Wooldridge, 2001. "Applications of Generalized Method of Moments Estimation," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 87-100, Fall.
    5. Lee, Min-Yang A. & Thunberg, Eric M., 2012. "An Inverse Demand System for New England Groundfish: Welfare Analysis of the Transition to Catch Share Management," 2012 Annual Meeting, August 12-14, 2012, Seattle, Washington 123879, Agricultural and Applied Economics Association.
    6. Carlsson, Mikael, 2000. "Measures of Technology and the Short-Run Responses to Technology Shocks - Is the RBC-Model Consistent with Swedish Manufacturing Data?," Working Paper Series 2000:20, Uppsala University, Department of Economics.
    7. Marcelo Moreira & Geert Ridder, 2019. "Efficiency loss of asymptotically efficient tests in an instrumental variables regression," CeMMAP working papers CWP03/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    8. de Castro, Luciano & Galvao, Antonio F. & Kaplan, David M. & Liu, Xin, 2019. "Smoothed GMM for quantile models," Journal of Econometrics, Elsevier, vol. 213(1), pages 121-144.
    9. Andrews, Donald W.K. & Cheng, Xu, 2013. "Maximum likelihood estimation and uniform inference with sporadic identification failure," Journal of Econometrics, Elsevier, vol. 173(1), pages 36-56.
    10. Jushan Bai & Sung Hoon Choi & Yuan Liao, 2021. "Feasible generalized least squares for panel data with cross-sectional and serial correlations," Empirical Economics, Springer, vol. 60(1), pages 309-326, January.
    11. Nathan H. Miller & Matthew Osborne, 2014. "Spatial differentiation and price discrimination in the cement industry: evidence from a structural model," RAND Journal of Economics, RAND Corporation, vol. 45(2), pages 221-247, June.
    12. Masih, A. Mansur M. & Masih, Rumi, 2002. "Propagative causal price transmission among international stock markets: evidence from the pre- and postglobalization period," Global Finance Journal, Elsevier, vol. 13(1), pages 63-91.
    13. Kitagawa, Toru & Montiel Olea, José Luis & Payne, Jonathan & Velez, Amilcar, 2020. "Posterior distribution of nondifferentiable functions," Journal of Econometrics, Elsevier, vol. 217(1), pages 161-175.
    14. Martínez-Iriarte, Julián & Sun, Yixiao & Wang, Xuexin, 2020. "Asymptotic F tests under possibly weak identification," Journal of Econometrics, Elsevier, vol. 218(1), pages 140-177.
    15. Maria Björklund & Mikael Carlsson & Oskar Nordström Skans, 2019. "Fixed-Wage Contracts and Monetary Non-neutrality," American Economic Journal: Macroeconomics, American Economic Association, vol. 11(2), pages 171-192, April.
    16. Jan F. Kiviet & Qu Feng, 2014. "Efficiency Gains by Modifying GMM Estimation in Linear Models under Heteroskedasticity," UvA-Econometrics Working Papers 14-06, Universiteit van Amsterdam, Dept. of Econometrics.
    17. Lise Pichette, 2004. "Are Wealth Effects Important for Canada," Bank of Canada Review, Bank of Canada, vol. 2004(Spring), pages 29-35.
    18. Ketz, Philipp, 2018. "Subvector inference when the true parameter vector may be near or at the boundary," Journal of Econometrics, Elsevier, vol. 207(2), pages 285-306.
    19. Andrews, Donald W.K. & Cheng, Xu, 2014. "Gmm Estimation And Uniform Subvector Inference With Possible Identification Failure," Econometric Theory, Cambridge University Press, vol. 30(2), pages 287-333, April.
    20. Andrews, Donald W.K. & Cheng, Xu & Guggenberger, Patrik, 2020. "Generic results for establishing the asymptotic size of confidence sets and tests," Journal of Econometrics, Elsevier, vol. 218(2), pages 496-531.

    More about this item

    Keywords

    confidence intervals; simulation; structural models; policy effects;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ifs:cemmap:23/13. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Emma Hyman (email available below). General contact details of provider: https://edirc.repec.org/data/cmifsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.