IDEAS home Printed from https://ideas.repec.org/a/wly/apecpp/v43y2021i4p1330-1351.html
   My bibliography  Save this article

The evolving ethics of analysis, publication, and transparency in applied economics

Author

Listed:
  • Travis J. Lybbert
  • Steven T. Buccola

Abstract

Credible economic research demands discipline and defensible modeling assumptions—both theoretical and empirical—but incentives to strategically shape findings (e.g., p‐hack) can be strong. We examine recent waves of empiricism in economics and the ethical concerns and responses they prompted. Statistical abuses that opportunistically search for significance are often inseparable from conceptual abuses of opportunistic model identification (i.e., p‐hacking writ large). We compare neoclassical with positivist hacking proclivities and explore associated implications for empirical analysis and peer review. Drawing on our experiences, 25 years apart, as AJAE editors we reflect on efforts to evaluate research quality and enhance research transparency.

Suggested Citation

  • Travis J. Lybbert & Steven T. Buccola, 2021. "The evolving ethics of analysis, publication, and transparency in applied economics," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(4), pages 1330-1351, December.
  • Handle: RePEc:wly:apecpp:v:43:y:2021:i:4:p:1330-1351
    DOI: 10.1002/aepp.13131
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/aepp.13131
    Download Restriction: no

    File URL: https://libkey.io/10.1002/aepp.13131?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Angus Deaton, 2020. "Randomization in the Tropics Revisited: a Theme and Eleven Variations," NBER Working Papers 27600, National Bureau of Economic Research, Inc.
    2. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    3. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    4. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    5. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    6. repec:bla:jecsur:v:16:y:2002:i:4:p:569-89 is not listed on IDEAS
    7. Asim I. Khwaja & Kunal Mangal, 2018. "Review of Economics and Statistics over the Past 100 Years: Content," The Review of Economics and Statistics, MIT Press, vol. 100(4), pages 1-1, October.
    8. John Gibson, 2019. "Are You Estimating the Right Thing? An Editor Reflects," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 41(3), pages 329-350.
    9. Burlig, Fiona, 2018. "Improving transparency in observational social science research: A pre-analysis plan approach," Economics Letters, Elsevier, vol. 168(C), pages 56-60.
    10. Christopher B. Barrett & Michael R. Carter, 2010. "The Power and Pitfalls of Experiments in Development Economics: Some Non-random Reflections," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 32(4), pages 515-548.
    11. Josephson, Anna & Michler, Jeffrey D., 2018. "Viewpoint: Beasts of the field? Ethics in agricultural and applied economics," Food Policy, Elsevier, vol. 79(C), pages 1-11.
    12. Fumagalli, Roberto, 2013. "The Futile Search For True Utility," Economics and Philosophy, Cambridge University Press, vol. 29(3), pages 325-347, November.
    13. Bruce L. Gardner, 1992. "How the Data We Make Can Unmake Us: Annals of Factology," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 74(5), pages 1066-1075.
    14. Arthur Lewbel, 2019. "The Identification Zoo: Meanings of Identification in Econometrics," Journal of Economic Literature, American Economic Association, vol. 57(4), pages 835-903, December.
    15. Peter E. Kennedy, 2002. "Sinning in the Basement: What are the Rules? The Ten Commandments of Applied Econometrics," Journal of Economic Surveys, Wiley Blackwell, vol. 16(4), pages 569-589, September.
    16. Valentin Amrhein & Sander Greenland & Blake McShane, 2019. "Scientists rise up against statistical significance," Nature, Nature, vol. 567(7748), pages 305-307, March.
    17. Guala, Francesco, 2019. "Preferences: neither behavioural nor mental," Economics and Philosophy, Cambridge University Press, vol. 35(3), pages 383-401, November.
    18. Angrist, Joshua D. & Krueger, Alan B., 1999. "Empirical strategies in labor economics," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 23, pages 1277-1366, Elsevier.
    19. Mas-Colell, Andreu & Whinston, Michael D. & Green, Jerry R., 1995. "Microeconomic Theory," OUP Catalogue, Oxford University Press, number 9780195102680.
    20. Paul Mizen (ed.), 2003. "Central Banking, Monetary Theory and Practice," Books, Edward Elgar Publishing, number 2329.
    21. John Ioannidis & Chris Doucouliagos, 2013. "What'S To Know About The Credibility Of Empirical Economics?," Journal of Economic Surveys, Wiley Blackwell, vol. 27(5), pages 997-1004, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Abel Brodeur & Scott Carrell & David Figlio & Lester Lusher, 2023. "Unpacking P-hacking and Publication Bias," American Economic Review, American Economic Association, vol. 113(11), pages 2974-3002, November.
    2. Sarah A. Janzen & Jeffrey D. Michler, 2021. "Ulysses' pact or Ulysses' raft: Using pre‐analysis plans in experimental and nonexperimental research," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(4), pages 1286-1304, December.
    3. Vecchio, Riccardo, 2022. "Fostering the debate among scholars to support the advancement of knowledge in the food-related consumer research: A commentary," Economia agro-alimentare / Food Economy, Italian Society of Agri-food Economics/Società Italiana di Economia Agro-Alimentare (SIEA), vol. 24(2), September.
    4. Riccardo Vecchio, 2022. "Fostering the debate among scholars to support the advancement of knowledge in the food-related consumer research: A commentary," Economia agro-alimentare, FrancoAngeli Editore, vol. 24(2), pages 1-15.
    5. Jeffrey D. Michler & William A. Masters & Anna Josephson, 2021. "Research ethics beyond the IRB: Selection bias and the direction of innovation in applied economics," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(4), pages 1352-1365, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    2. Josephson, Anna & Michler, Jeffrey D., 2018. "Viewpoint: Beasts of the field? Ethics in agricultural and applied economics," Food Policy, Elsevier, vol. 79(C), pages 1-11.
    3. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    4. Moscati, Ivan, 2021. "On the recent philosophy of decision theory," LSE Research Online Documents on Economics 115039, London School of Economics and Political Science, LSE Library.
    5. Ankel-Peters, Jörg & Schmidt, Christoph M., 2023. "Rural electrification, the credibility revolution, and the limits of evidence-based policy," Ruhr Economic Papers 1051, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    7. Duo Qin & Sophie van H¸llen & Qing-Chao Wang, 2014. "What Happens to Wage Elasticities When We Strip Playometrics? Revisiting Married Women Labour Supply Model," Working Papers 190, Department of Economics, SOAS University of London, UK.
    8. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
    9. Burt S. Barnow & Jeffrey Smith, 2015. "Employment and Training Programs," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 2, pages 127-234, National Bureau of Economic Research, Inc.
    10. Chowdhury, Rosen & Cook, Steve & Watson, Duncan, 2023. "Reconsidering the relationship between health and income in the UK," Social Science & Medicine, Elsevier, vol. 332(C).
    11. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    12. Adam Gorajek & Joel Bank & Andrew Staib & Benjamin Malin & Hamish Fitchett, 2021. "Star Wars at Central Banks," RBA Research Discussion Papers rdp2021-02, Reserve Bank of Australia.
    13. Gechert, Sebastian & Mey, Bianka & Opatrny, Matej & Havranek, Tomas & Stanley, T. D. & Bom, Pedro R. D. & Doucouliagos, Hristos & Heimberger, Philipp & Irsova, Zuzana & Rachinger, Heiko J., 2023. "Conventional Wisdom, Meta-Analysis, and Research Revision in Economics," EconStor Preprints 280745, ZBW - Leibniz Information Centre for Economics.
    14. Steven T. Berry & Philip A. Haile, 2024. "Nonparametric Identification of Differentiated Products Demand Using Micro Data," Econometrica, Econometric Society, vol. 92(4), pages 1135-1162, July.
    15. Michael Lechner, 2023. "Causal Machine Learning and its use for public policy," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 159(1), pages 1-15, December.
    16. Nathan Canen & Kristopher Ramsay, 2024. "Quantifying theory in politics: Identification, interpretation, and the role of structural methods," Journal of Theoretical Politics, , vol. 36(4), pages 301-327, October.
    17. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    18. Giovanni Peri & Vasil Yasenov, 2019. "The Labor Market Effects of a Refugee Wave: Synthetic Control Method Meets the Mariel Boatlift," Journal of Human Resources, University of Wisconsin Press, vol. 54(2), pages 267-309.
    19. Alecos Papadopoulos, 2022. "Trade liberalization and growth: a quantile moderator for Hoyos’ (2021) replication study of Estevadeordal and Taylor (2013)," Empirical Economics, Springer, vol. 63(1), pages 549-563, July.
    20. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:apecpp:v:43:y:2021:i:4:p:1330-1351. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)2040-5804 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.