IDEAS home Printed from https://ideas.repec.org/p/zbw/i4rdps/270.html

Seeking Scientific Consensus - An Expert Survey on the Replication Debate between Acemoglu et al. (2001) and Albouy (2012)

Author

Listed:
  • Buchner, Martin
  • Rose, Julian
  • Johannesson, Magnus
  • Malan, Mandy
  • Ankel-Peters, Jörg

Abstract

Consensus is crucial to authoritative science, as is replicability. Yet, in economics and the social sciences, the publication of contradictory replications often sparks fierce debates between replicators and original authors. This paper investigates whether experts can reach a consensus on a famous yet unsettled debate about the robustness of the seminal paper by Acemoglu, Johnson, and Robinson (AJR, 2001) following a replication by Albouy (2012). We recruited 352 experts mainly from the pool of scholars citing one of the involved or similar articles. Through a structured online questionnaire, we assess the extent to which these experts align with AJR or Albouy. Our findings indicate no consensus on whether the original results hold after Albouy's replication, although there is a slight tendency among experts to side with the replicator. Exploratory heterogeneity analysis suggests that experts with greater academic credentials are more likely to align with Albouy. Our study demonstrates a potential way to scope scientific consensus formation and navigate replication debates and contested literatures.

Suggested Citation

  • Buchner, Martin & Rose, Julian & Johannesson, Magnus & Malan, Mandy & Ankel-Peters, Jörg, 2025. "Seeking Scientific Consensus - An Expert Survey on the Replication Debate between Acemoglu et al. (2001) and Albouy (2012)," I4R Discussion Paper Series 270, The Institute for Replication (I4R).
    • Handle: RePEc:zbw:i4rdps:270
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/328050/1/I4R-DP270.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    2. Andrei Markevich & Ekaterina Zhuravskaya, 2018. "The Economic Effects of the Abolition of Serfdom: Evidence from the Russian Empire," American Economic Review, American Economic Association, vol. 108(4-5), pages 1074-1117, April.
    3. Joshua D. Angrist & Alan B. Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 69-85, Fall.
    4. McCloskey, Donald N, 1983. "The Rhetoric of Economics," Journal of Economic Literature, American Economic Association, vol. 21(2), pages 481-517, June.
    5. Freese, Jeremy & Peterson, David, 2017. "Replication in Social Science," SocArXiv 5bck9, Center for Open Science.
    6. Casey, Gregory & Klemp, Marc, 2021. "Historical instruments and contemporary endogenous regressors," Journal of Development Economics, Elsevier, vol. 149(C).
    7. Hannah Fraser & Martin Bush & Bonnie C Wintle & Fallon Mody & Eden T Smith & Anca M Hanea & Elliot Gould & Victoria Hemming & Daniel G Hamilton & Libby Rumpff & David P Wilkinson & Ross Pearson & Feli, 2023. "Predicting reliability through structured expert elicitation with the repliCATS (Collaborative Assessments for Trustworthy Science) process," PLOS ONE, Public Library of Science, vol. 18(1), pages 1-16, January.
    8. David Y. Albouy, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Comment," American Economic Review, American Economic Association, vol. 102(6), pages 3059-3076, October.
    9. Conley, Timothy G. & Kelly, Morgan, 2025. "The standard errors of persistence," Journal of International Economics, Elsevier, vol. 153(C).
    10. Guido Tabellini, 2010. "Culture and Institutions: Economic Development in the Regions of Europe," Journal of the European Economic Association, European Economic Association, vol. 8(4), pages 677-716, June.
    11. Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018. "Redefine statistical significance," Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
      • Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.
    12. Brodeur, Abel & Esterling, Kevin & Ankel-Peters, Jörg & Bueno, Natália S & Desposato, Scott & Dreber, Anna & Genovese, Federica & Green, Donald P & Hepplewhite, Matthew & de la Guardia, Fernando Hoces, 2024. "Promoting Reproducibility and Replicability in Political Science," Department of Economics, Working Paper Series qt23n3n3dg, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    13. Jutta Bolt & Dirk Bezemer, 2009. "Understanding Long-Run African Growth: Colonial Institutions or Colonial Education?," Journal of Development Studies, Taylor & Francis Journals, vol. 45(1), pages 24-54.
    14. Owen Ozier, 2021. "Replication Redux: The Reproducibility Crisis and the Case of Deworming [Economics of Mass Deworming Programs]," The World Bank Research Observer, World Bank, vol. 36(1), pages 101-130.
    15. repec:hal:spmain:info:hdl:2441/67ft27s7u58ocangahl1jigu6p is not listed on IDEAS
    16. Willy Aspinall, 2010. "A route to more tractable expert advice," Nature, Nature, vol. 463(7279), pages 294-295, January.
    17. Alberto Alesina & Paola Giuliano, 2015. "Culture and Institutions," Journal of Economic Literature, American Economic Association, vol. 53(4), pages 898-944, December.
    18. Edward L. Glaeser & Rafael La Porta & Florencio Lopez-de-Silanes & Andrei Shleifer, 2004. "Do Institutions Cause Growth?," Journal of Economic Growth, Springer, vol. 9(3), pages 271-303, September.
    19. repec:spo:wpmain:info:hdl:2441/67ft27s7u58ocangahl1jigu6p is not listed on IDEAS
    20. Marion Fourcade & Etienne Ollion & Yann Algan, 2015. "La superioridad de los economistas," Revista de Economía Institucional, Universidad Externado de Colombia - Facultad de Economía, vol. 17(33), pages 13-43, July-Dece.
    21. John Luke Gallup & Jeffrey D. Sachs & Andrew Mellinger, 1999. "Geography and Economic Development," CID Working Papers 1, Center for International Development at Harvard University.
    22. M. Fourcade & E. Ollion & Y. Algan, 2015. "The Superiority of Economists," Voprosy Ekonomiki, NP Voprosy Ekonomiki, issue 7.
    23. William Easterly & Ross Levine, 2016. "The European origins of economic development," Journal of Economic Growth, Springer, vol. 21(3), pages 225-257, September.
    24. Frey, Bruno S, et al, 1984. "Consensus and Dissension among Economists: An Empirical Inquiry," American Economic Review, American Economic Association, vol. 74(5), pages 986-994, December.
    25. repec:fth:prinin:455 is not listed on IDEAS
    26. Daron Acemoglu & Simon Johnson & James A. Robinson, 2001. "The Colonial Origins of Comparative Development: An Empirical Investigation," American Economic Review, American Economic Association, vol. 91(5), pages 1369-1401, December.
    27. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    28. Amir Rubin & Eran Rubin, 2021. "Systematic Bias in the Progress of Research," Journal of Political Economy, University of Chicago Press, vol. 129(9), pages 2666-2719.
    29. Teplitskiy, Misha & Duede, Eamon & Menietti, Michael & Lakhani, Karim R., 2022. "How status of research papers affects the way they are read and cited," Research Policy, Elsevier, vol. 51(4).
    30. John Luke Gallup & Jeffrey D. Sachs & Andrew Mellinger, 1999. "Geography and Economic Development," CID Working Papers 01A, Center for International Development at Harvard University.
    31. Forsell, Eskil & Viganola, Domenico & Pfeiffer, Thomas & Almenberg, Johan & Wilson, Brad & Chen, Yiling & Nosek, Brian A. & Johannesson, Magnus & Dreber, Anna, 2019. "Predicting replication outcomes in the Many Labs 2 study," Journal of Economic Psychology, Elsevier, vol. 75(PA).
    32. Gallup, John & Sachs, Jeffrey, 1999. "Geography and Economic Development," Harvard Institute for International Development (HIID) Papers 294434, Harvard University, Kennedy School of Government.
    33. Daron Acemoglu & Simon Johnson & James A. Robinson, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Reply," American Economic Review, American Economic Association, vol. 102(6), pages 3077-3110, October.
    34. Gallup, J.L. & Sachs, J.D. & Mullinger, A., 1999. "Geography and Economic Development," Papers 1, Chicago - Graduate School of Business.
    35. Claudius Gräbner & Birte Strunk, 2020. "Pluralism in economics: its critiques and their lessons," Journal of Economic Methodology, Taylor & Francis Journals, vol. 27(4), pages 311-329, October.
    36. Joshua Angrist & Alan Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Working Papers 834, Princeton University, Department of Economics, Industrial Relations Section..
    37. M. Fourcade & E. Ollion & Y. Algan., 2015. "The Superiority of Economists," VOPROSY ECONOMIKI, N.P. Redaktsiya zhurnala "Voprosy Economiki", vol. 7.
    38. Dan A. Black & Seth G. Sanders & Evan J. Taylor & Lowell J. Taylor, 2015. "The Impact of the Great Migration on Mortality of African Americans: Evidence from the Deep South," American Economic Review, American Economic Association, vol. 105(2), pages 477-503, February.
    39. John W. McArthur & Jeffrey D. Sachs, 2001. "Institutions and Geography: Comment on Acemoglu, Johnson and Robinson (2000)," NBER Working Papers 8114, National Bureau of Economic Research, Inc.
    40. Norman Dalkey & Olaf Helmer, 1963. "An Experimental Application of the DELPHI Method to the Use of Experts," Management Science, INFORMS, vol. 9(3), pages 458-467, April.
    41. repec:osf:socarx:5bck9_v1 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Braunfels, Elias, 2016. "Further Unbundling Institutions," Discussion Paper Series in Economics 13/2016, Norwegian School of Economics, Department of Economics.
    2. Bennett, Daniel L. & Faria, Hugo J. & Gwartney, James D. & Morales, Daniel R., 2017. "Economic Institutions and Comparative Economic Development: A Post-Colonial Perspective," World Development, Elsevier, vol. 96(C), pages 503-519.
    3. Jubril Animashaun & Ada Wossink & Katsushi S. Imai, 2023. "Colonialism, Institutional Quality, and the Resource Curse," Discussion Paper Series DP2023-19, Research Institute for Economics & Business Administration, Kobe University.
    4. Valentina A Assenova & Matthew Regele, 2017. "Revisiting the effect of colonial institutions on comparative economic development," PLOS ONE, Public Library of Science, vol. 12(5), pages 1-16, May.
    5. Natkhov, T. & Polishchuk, L., 2017. "Political Economy of Institutions and Development: The Importance of Being Inclusive. Reflection on "Why Nations Fail" by D. Acemoglu and J. Robinson. Part I. Institutions and Economic Devel," Journal of the New Economic Association, New Economic Association, vol. 34(2), pages 12-38.
    6. Stimpfle, Alexander & Stadelmann, David, 2015. "The Impact of Fundamental Development Factors on Different Income Groups: International Evidence," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113128, Verein für Socialpolitik / German Economic Association.
    7. Kyriacou, Andreas P., 2016. "Individualism–collectivism, governance and economic development," European Journal of Political Economy, Elsevier, vol. 42(C), pages 91-104.
    8. Tang, Sam Hak Kan & Groenewold, Nicolaas & Leung, Charles Ka Yui, 2008. "The link between institutions, technical change and macroeconomic volatility," Journal of Macroeconomics, Elsevier, vol. 30(4), pages 1520-1549, December.
    9. Ho, Chi Pui, 2016. "GeoPopulation-Institution Hypothesis: Reconciling American Development Process and Reversal of Fortune within a Unified Growth Framework," MPRA Paper 73863, University Library of Munich, Germany.
    10. James B. Ang, 2015. "Agricultural Legacy, Individualistic Culture, and Techology Adoption," Economic Growth Centre Working Paper Series 1506, Nanyang Technological University, School of Social Sciences, Economic Growth Centre.
    11. Mauricio Cárdenas, 2007. "Economic Growth in Colombia : a reversal of "fortune"?," Revista ESPE - Ensayos sobre Política Económica, Banco de la Republica de Colombia, vol. 25(53), pages 220-259, January.
    12. Madhoo, Yeti Nisha & Nath, Shyam, 2025. "Colonial education disparity, contemporary institutions, and long-run economic performance," World Development Perspectives, Elsevier, vol. 39(C).
    13. Richard S.J. Tol, 2025. "Climate Determinism Reborn," Working Paper Series 0725, Department of Economics, University of Sussex Business School.
    14. Casey, Gregory & Klemp, Marc, 2021. "Historical instruments and contemporary endogenous regressors," Journal of Development Economics, Elsevier, vol. 149(C).
    15. Messono, Omang & Asongu, Simplice, 2021. "Historical Prevalence of Infectious Diseases and Entrepreneurship: the Role of Institutions in 125 Countries," MPRA Paper 111842, University Library of Munich, Germany.
    16. P. Dorian Owen, 2017. "Evaluating Ingenious Instruments for Fundamental Determinants of Long-Run Economic Growth and Development," Econometrics, MDPI, vol. 5(3), pages 1-33, September.
    17. Glawe, Linda & Wagner, Helmut, 2017. "The Deep Determinants of the Middle-Income Trap," CEAMeS Discussion Paper Series 10/2017, University of Hagen, Center for East Asia Macro-economic Studies (CEAMeS), revised 2017.
    18. Fuchs-Schündeln, N. & Hassan, T.A., 2016. "Natural Experiments in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 923-1012, Elsevier.
    19. James Feyrer, 2019. "Trade and Income—Exploiting Time Series in Geography," American Economic Journal: Applied Economics, American Economic Association, vol. 11(4), pages 1-35, October.
    20. Erkan Goeren, 2017. "The Role of Novelty-Seeking Traits in Contemporary Knowledge Creation," Working Papers V-402-17, University of Oldenburg, Department of Economics, revised Sep 2017.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:i4rdps:270. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://www.i4replication.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.