IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v49y2025i5p799-813.html

Dubious Evidence, Valuable Information

Author

Listed:
  • Eugene Bardach

Abstract

I explore how “low-quality evidence†from program performance might still be useful in decision-making. Conceptually, a local government named “Here†is motivated to consider a program from “Elsewhere†that seems to show year-over-year exemplary performance. Here must manage five sources of uncertainty about whether and how to extrapolate from Elsewhere: chance in assessing Elsewhere’s performance; illusion due to confounding variables; estimating the several powers of the program’s components; substitutions in the design process made by Elsewhere and contemplated by Here; and estimating whether in the final analysis Here can meet its own breakeven criterion for going ahead. Here can begin with Elsewhere’s experience, but it still must do much thinking and information-collecting on its own.

Suggested Citation

  • Eugene Bardach, 2025. "Dubious Evidence, Valuable Information," Evaluation Review, , vol. 49(5), pages 799-813, October.
    • Handle: RePEc:sae:evarev:v:49:y:2025:i:5:p:799-813
    DOI: 10.1177/0193841X251324594
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X251324594
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X251324594?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Thomas D. Cook, 2014. "Generalizing Causal Knowledge In The Policy Sciences: External Validity As A Task Of Both Multiattribute Representation And Multiattribute Extrapolation," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 527-536, March.
    2. Stefano DellaVigna & Woojin Kim & Elizabeth Linos, 2024. "Bottlenecks for Evidence Adoption," Journal of Political Economy, University of Chicago Press, vol. 132(8), pages 2748-2789.
    3. Duflo, Esther, 2017. "The Economist as Plumber," CEPR Discussion Papers 11881, C.E.P.R. Discussion Papers.
    4. Esther Duflo, 2017. "Richard T. Ely Lecture: The Economist as Plumber," American Economic Review, American Economic Association, vol. 107(5), pages 1-26, May.
    5. Esther Duflo, 2017. "The Economist as Plumber," NBER Working Papers 23213, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:osf:osfxxx:nwp8k_v1 is not listed on IDEAS
    2. Krause, Philipp & Hernández Licona, Gonzalo, 2020. "From experimental findings to evidence-based policy," World Development, Elsevier, vol. 127(C).
    3. repec:hal:cepnwp:hal-03139457 is not listed on IDEAS
    4. Rehse, Dominik & Tremöhlen, Felix, 2020. "Fostering participation in digital public health interventions: The case of digital contact tracing," ZEW Discussion Papers 20-076, ZEW - Leibniz Centre for European Economic Research.
    5. Wei Yang Tham & Joseph Staudt & Elisabeth Ruth Perlman & Stephanie D. Cheng, 2024. "Scientific Talent Leaks Out of Funding Gaps," Working Papers 24-08, Center for Economic Studies, U.S. Census Bureau.
    6. Peter Boettke & John Kroencke, 2020. "The real purpose of the program: a case study in James M. Buchanan’s efforts at academic entrepreneurship to “save the books” in economics," Public Choice, Springer, vol. 183(3), pages 227-245, June.
    7. Button, Kenneth & Frye, Hailey & Reaves, David, 2020. "Economic regulation and E-scooter networks in the USA," Research in Transportation Economics, Elsevier, vol. 84(C).
    8. Kvangraven, Ingrid Harvold, 2025. "Cracks in the “gold standard”: The Eurocentrism of mining in development economics," World Development, Elsevier, vol. 192(C).
    9. Boris Salazar-Trujillo & Daniel Otero Robles, 2019. "La revolución empírica en economía," Apuntes del Cenes, Universidad Pedagógica y Tecnológica de Colombia, vol. 38(68), pages 15-48.
    10. Masselus, Lise & Petrik, Christina & Ankel-Peters, Jörg, 2024. "Lost in the design space? Construct validity in the microfinance literature," Ruhr Economic Papers 1097, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    11. Elizabeth Linos & Sanaz Mobasseri & Nina Roussille, 2025. "Intersectional Peer Effects at Work: The Effect of White Coworkers on Black Women’s Careers," Management Science, INFORMS, vol. 71(9), pages 7600-7630, September.
    12. Dimo Dimov & Markku Maula & A. Georges L. Romme, 2023. "Crafting and Assessing Design Science Research for Entrepreneurship," Entrepreneurship Theory and Practice, , vol. 47(5), pages 1543-1567, September.
    13. Löschel, Andreas & Großkurth, Philipp & Colombier, Michel & Criqui, Patrick & Xiangwan, Du & Frei, Christoph & Gethmann, Carl Friedrich & Gummer, John & Lecocq, Franck & Parikh, Jyoti K. & Sauer, Dirk, 2018. "Establishing an expert advisory commission to assist the G20's energy transformation processes," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy, vol. 12, pages 1-13.
    14. Justman, Moshe, 2018. "Randomized controlled trials informing public policy: Lessons from project STAR and class size reduction," European Journal of Political Economy, Elsevier, vol. 54(C), pages 167-174.
    15. Steve J. Bickley & Ho Fai Chan & Benno Torgler, 2022. "Artificial intelligence in the field of economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 2055-2084, April.
    16. Fonseca Morello, Thiago, 2022. "Subsidization of mechanized tillage as an alternative to fire-based land preparation by smallholders: An economic appraisal of the case of southwestern Brazilian Amazon," Land Use Policy, Elsevier, vol. 123(C).
    17. Rao, Vijayendra, 2020. "Evidence-based development needs a diversity of tools, with a bottom-up process of “embedded” dialogue," World Development, Elsevier, vol. 127(C).
    18. Saileshsingh Gunessee & Tom Lane, 2020. "Is Economics An Experimental Science? A Textbook Perspective," Discussion Papers 2020-16, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    19. Tatsuru Kikuchi, 2025. "Stochastic Boundaries in Spatial General Equilibrium: A Diffusion-Based Approach to Causal Inference with Spillover Effects," Papers 2508.06594, arXiv.org.
    20. Drèze, Jean, 2020. "Policy beyond evidence," World Development, Elsevier, vol. 127(C).
    21. Vollebergh, Herman, 2023. "Evaluating policy packages for a low-carbon transitions – Principles and applications," Other publications TiSEM 0a3c8aaf-2b01-4fe9-bfe7-a, Tilburg University, School of Economics and Management.
    22. Sergio Mariotti, 2022. "The economics–engineering nexus: response to the commentaries," Economia e Politica Industriale: Journal of Industrial and Business Economics, Springer;Associazione Amici di Economia e Politica Industriale, vol. 49(1), pages 1-29, March.

    More about this item

    Keywords

    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:49:y:2025:i:5:p:799-813. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.