IDEAS home Printed from
   My bibliography  Save this paper

Policy analysis with incredible certitude


  • Charles F. Manski

    () (Institute for Fiscal Studies and Northwestern University)


Analyses of public policy regularly express certitude about the consequences of alternative policy choices. Yet policy predictions often are fragile, with conclusions resting on critical unsupported assumptions or leaps of logic. Then the certitude of policy analysis is not credible. I develop a typology of incredible analytical practices and gives illustrative cases. I call these practices conventional certitude, dueling certitudes, conflating science and advocacy, wishful extrapolation, illogical certitude, and media overreach.

Suggested Citation

  • Charles F. Manski, 2011. "Policy analysis with incredible certitude," CeMMAP working papers CWP04/11, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  • Handle: RePEc:ifs:cemmap:04/11

    Download full text from publisher

    File URL:
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    1. Jeremy T. Fox & Amit Gandhi, 2009. "Identifying Heterogeneity in Economic Choice Models," NBER Working Papers 15147, National Bureau of Economic Research, Inc.
    2. Ruud H. Koning & Geert Ridder, 2003. "Discrete choice and stochastic utility maximization," Econometrics Journal, Royal Economic Society, vol. 6(1), pages 1-27, June.
    3. Chiappori, Pierre-André & Komunjer, Ivana & Kristensen, Dennis, 2015. "Nonparametric identification and estimation of transformation models," Journal of Econometrics, Elsevier, vol. 188(1), pages 22-39.
    4. Victor Chernozhukov & Sokbae Lee & Adam M. Rosen, 2013. "Intersection Bounds: Estimation and Inference," Econometrica, Econometric Society, vol. 81(2), pages 667-737, March.
    5. Lewbel, Arthur, 2000. "Semiparametric qualitative response model estimation with unknown heteroscedasticity or instrumental variables," Journal of Econometrics, Elsevier, vol. 97(1), pages 145-177, July.
    6. Ivar Ekeland & Alfred Galichon & Marc Henry, 2010. "Optimal transportation and the falsifiability of incompletely specified economic models," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 42(2), pages 355-374, February.
    7. Matzkin, Rosa L., 1993. "Nonparametric identification and estimation of polychotomous choice models," Journal of Econometrics, Elsevier, vol. 58(1-2), pages 137-168, July.
    8. Victor Chernozhukov & Han Hong & Elie Tamer, 2007. "Estimation and Confidence Regions for Parameter Sets in Econometric Models," Econometrica, Econometric Society, vol. 75(5), pages 1243-1284, September.
    9. Arie Beresteanu & Francesca Molinari, 2008. "Asymptotic Properties for a Class of Partially Identified Models," Econometrica, Econometric Society, vol. 76(4), pages 763-814, July.
    10. Rosen, Adam M., 2008. "Confidence sets for partially identified parameters that satisfy a finite number of moment inequalities," Journal of Econometrics, Elsevier, vol. 146(1), pages 107-117, September.
    11. Canay, Ivan A., 2010. "EL inference for partially identified models: Large deviations optimality and bootstrap validity," Journal of Econometrics, Elsevier, vol. 156(2), pages 408-425, June.
    12. Arie Beresteanu & Ilya Molchanov & Francesca Molinari, 2011. "Sharp Identification Regions in Models With Convex Moment Predictions," Econometrica, Econometric Society, vol. 79(6), pages 1785-1821, November.
    13. Galichon, Alfred & Henry, Marc, 2009. "A test of non-identifying restrictions and confidence regions for partially identified parameters," Journal of Econometrics, Elsevier, vol. 152(2), pages 186-196, October.
    14. Federico A. Bugni, 2010. "Bootstrap Inference in Partially Identified Models Defined by Moment Inequalities: Coverage of the Identified Set," Econometrica, Econometric Society, vol. 78(2), pages 735-753, March.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Chen, Qianying & Funke, Michael & Paetz, Michael, 2012. "Market and non-market monetary policy tools in a calibrated DSGE model for mainland China," BOFIT Discussion Papers 16/2012, Bank of Finland, Institute for Economies in Transition.
    2. Argentiero, Amedeo & Bovi, Maurizio & Cerqueti, Roy, 2015. "Over consumption. A horse race of Bayesian DSGE models," MPRA Paper 66445, University Library of Munich, Germany.
    3. Mester, Loretta J., 2016. "Acknowledging Uncertainty, 10-07-2016; Shadow Open Market Committee Fall Meeting, New York, NY," Speech 77, Federal Reserve Bank of Cleveland.
    4. Douglas W. Elmendorf, 2015. "“Dynamic Scoring”: Why and How to Include Macroeconomic Effects in Budget Estimates for Legislative Proposals," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 46(2 (Fall)), pages 91-149.
    5. Pesaran, M. Hashem & Smith, Ron P., 2011. "Beyond the DSGE Straitjacket," IZA Discussion Papers 5661, Institute for the Study of Labor (IZA).
    6. Hunt Allcott, 2012. "Site Selection Bias in Program Evaluation," NBER Working Papers 18373, National Bureau of Economic Research, Inc.
    7. Whittington, Dale & Jeuland, Marc & Barker, Kate & Yuen, Yvonne, 2012. "Setting Priorities, Targeting Subsidies among Water, Sanitation, and Preventive Health Interventions in Developing Countries," World Development, Elsevier, vol. 40(8), pages 1546-1568.
    8. Muller, Sean, 2014. "Randomised trials for policy: a review of the external validity of treatment effects," SALDRU Working Papers 127, Southern Africa Labour and Development Research Unit, University of Cape Town.
    9. Charles F. Manski, 2015. "Communicating Uncertainty in Official Economic Statistics: An Appraisal Fifty Years after Morgenstern," Journal of Economic Literature, American Economic Association, vol. 53(3), pages 631-653, September.
    10. Lawrence Mead, 2015. "Only connect: Why government often ignores research," Policy Sciences, Springer;Society of Policy Sciences, vol. 48(2), pages 257-272, June.
    11. Charles F. Manski, 2014. "Communicating Uncertainty in Official Economic Statistics," NBER Working Papers 20098, National Bureau of Economic Research, Inc.

    More about this item

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate
    • H68 - Public Economics - - National Budget, Deficit, and Debt - - - Forecasts of Budgets, Deficits, and Debt

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ifs:cemmap:04/11. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Emma Hyman). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.