IDEAS home Printed from https://ideas.repec.org/p/zbw/rwirep/330180.html
   My bibliography  Save this paper

Seeking scientific consensus: An expert survey on the replication debate between Acemoglu et al. (2001) and Albouy (2012)

Author

Listed:
  • Buchner, Martin
  • Rose, Julian
  • Johannesson, Magnus
  • Malan, Mandy
  • Ankel-Peters, Jörg

Abstract

Consensus is crucial to authoritative science, as is replicability. Yet, in economics and the social sciences, the publication of contradictory replications often sparks fierce debates between replicators and original authors. This paper investigates whether experts can reach a consensus on a famous yet unsettled debate about the robustness of the seminal paper by Acemoglu, Johnson, and Robinson (AJR, 2001) following a replication by Albouy (2012). We recruited 352 experts mainly from the pool of scholars citing one of the involved or similar articles. Through a structured online questionnaire, we assess the extent to which these experts align with AJR or Albouy. Our findings indicate no consensus on whether the original results hold after Albouy's replication, although there is a slight tendency among experts to side with the replicator. Exploratory heterogeneity analysis suggests that experts with greater academic credentials are more likely to align with Albouy. Our study demonstrates a potential way to scope scientific consensus formation and navigate replication debates and contested literatures.

Suggested Citation

  • Buchner, Martin & Rose, Julian & Johannesson, Magnus & Malan, Mandy & Ankel-Peters, Jörg, 2025. "Seeking scientific consensus: An expert survey on the replication debate between Acemoglu et al. (2001) and Albouy (2012)," Ruhr Economic Papers 1176, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    • Handle: RePEc:zbw:rwirep:330180
    DOI: 10.4419/96973361
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/330180/1/1939367859.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.4419/96973361?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joshua D. Angrist & Alan B. Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 69-85, Fall.
    2. Casey, Gregory & Klemp, Marc, 2021. "Historical instruments and contemporary endogenous regressors," Journal of Development Economics, Elsevier, vol. 149(C).
    3. Owen Ozier, 2021. "Replication Redux: The Reproducibility Crisis and the Case of Deworming [Economics of Mass Deworming Programs]," The World Bank Research Observer, World Bank, vol. 36(1), pages 101-130.
    4. John Luke Gallup & Jeffrey D. Sachs & Andrew Mellinger, 1999. "Geography and Economic Development," CID Working Papers 1, Center for International Development at Harvard University.
    5. Guido Tabellini, 2010. "Culture and Institutions: Economic Development in the Regions of Europe," Journal of the European Economic Association, European Economic Association, vol. 8(4), pages 677-716, June.
    6. Marion Fourcade & Etienne Ollion & Yann Algan, 2015. "The Superiority of Economists," Journal of Economic Perspectives, American Economic Association, vol. 29(1), pages 89-114, Winter.
    7. William Easterly & Ross Levine, 2016. "The European origins of economic development," Journal of Economic Growth, Springer, vol. 21(3), pages 225-257, September.
    8. Andrei Markevich & Ekaterina Zhuravskaya, 2018. "The Economic Effects of the Abolition of Serfdom: Evidence from the Russian Empire," American Economic Review, American Economic Association, vol. 108(4-5), pages 1074-1117, April.
    9. Daron Acemoglu & Simon Johnson & James A. Robinson, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Reply," American Economic Review, American Economic Association, vol. 102(6), pages 3077-3110, October.
    10. Marion Fourcade & Etienne Ollion & Yann Algan, 2015. "The Superiority of Economists," Journal of Economic Perspectives, American Economic Association, vol. 29(1), pages 89-114, Winter.
    11. Norman Dalkey & Olaf Helmer, 1963. "An Experimental Application of the DELPHI Method to the Use of Experts," Management Science, INFORMS, vol. 9(3), pages 458-467, April.
    12. McCloskey, Donald N, 1983. "The Rhetoric of Economics," Journal of Economic Literature, American Economic Association, vol. 21(2), pages 481-517, June.
    13. repec:fth:prinin:455 is not listed on IDEAS
    14. Forsell, Eskil & Viganola, Domenico & Pfeiffer, Thomas & Almenberg, Johan & Wilson, Brad & Chen, Yiling & Nosek, Brian A. & Johannesson, Magnus & Dreber, Anna, 2019. "Predicting replication outcomes in the Many Labs 2 study," Journal of Economic Psychology, Elsevier, vol. 75(PA).
    15. Brodeur, Abel & Esterling, Kevin & Ankel-Peters, Jörg & Bueno, Natália S. & Desposato, Scott & Dreber, Anna & Genovese, Federica & Green, Donald P. & Hepplewhite, Matthew & Hoces de la Guardia, Fernan, 2024. "Promoting Reproducibility and Replicability in Political Science," I4R Discussion Paper Series 100, The Institute for Replication (I4R).
    16. John W. McArthur & Jeffrey D. Sachs, 2001. "Institutions and Geography: Comment on Acemoglu, Johnson and Robinson (2000)," NBER Working Papers 8114, National Bureau of Economic Research, Inc.
    17. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    18. Hannah Fraser & Martin Bush & Bonnie C Wintle & Fallon Mody & Eden T Smith & Anca M Hanea & Elliot Gould & Victoria Hemming & Daniel G Hamilton & Libby Rumpff & David P Wilkinson & Ross Pearson & Feli, 2023. "Predicting reliability through structured expert elicitation with the repliCATS (Collaborative Assessments for Trustworthy Science) process," PLOS ONE, Public Library of Science, vol. 18(1), pages 1-16, January.
    19. David Y. Albouy, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Comment," American Economic Review, American Economic Association, vol. 102(6), pages 3059-3076, October.
    20. Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018. "Redefine statistical significance," Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
      • Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.
    21. Jutta Bolt & Dirk Bezemer, 2009. "Understanding Long-Run African Growth: Colonial Institutions or Colonial Education?," Journal of Development Studies, Taylor & Francis Journals, vol. 45(1), pages 24-54.
    22. repec:hal:spmain:info:hdl:2441/67ft27s7u58ocangahl1jigu6p is not listed on IDEAS
    23. Daron Acemoglu & Simon Johnson & James A. Robinson, 2001. "The Colonial Origins of Comparative Development: An Empirical Investigation," American Economic Review, American Economic Association, vol. 91(5), pages 1369-1401, December.
    24. Amir Rubin & Eran Rubin, 2021. "Systematic Bias in the Progress of Research," Journal of Political Economy, University of Chicago Press, vol. 129(9), pages 2666-2719.
    25. Teplitskiy, Misha & Duede, Eamon & Menietti, Michael & Lakhani, Karim R., 2022. "How status of research papers affects the way they are read and cited," Research Policy, Elsevier, vol. 51(4).
    26. John Luke Gallup & Jeffrey D. Sachs & Andrew Mellinger, 1999. "Geography and Economic Development," CID Working Papers 01A, Center for International Development at Harvard University.
    27. Gallup, J.L. & Sachs, J.D. & Mullinger, A., 1999. "Geography and Economic Development," Papers 1, Chicago - Graduate School of Business.
    28. Alberto Alesina & Paola Giuliano, 2015. "Culture and Institutions," Journal of Economic Literature, American Economic Association, vol. 53(4), pages 898-944, December.
    29. M. Fourcade & E. Ollion & Y. Algan., 2015. "The Superiority of Economists," VOPROSY ECONOMIKI, N.P. Redaktsiya zhurnala "Voprosy Economiki", vol. 7.
    30. Edward L. Glaeser & Rafael La Porta & Florencio Lopez-de-Silanes & Andrei Shleifer, 2004. "Do Institutions Cause Growth?," Journal of Economic Growth, Springer, vol. 9(3), pages 271-303, September.
    31. Freese, Jeremy & Peterson, David, 2017. "Replication in Social Science," SocArXiv 5bck9, Center for Open Science.
    32. Conley, Timothy G. & Kelly, Morgan, 2025. "The standard errors of persistence," Journal of International Economics, Elsevier, vol. 153(C).
    33. Willy Aspinall, 2010. "A route to more tractable expert advice," Nature, Nature, vol. 463(7279), pages 294-295, January.
    34. Marion Fourcade & Etienne Ollion & Yann Algan, 2015. "La superioridad de los economistas," Revista de Economía Institucional, Universidad Externado de Colombia - Facultad de Economía, vol. 17(33), pages 13-43, July-Dece.
    35. Frey, Bruno S, et al, 1984. "Consensus and Dissension among Economists: An Empirical Inquiry," American Economic Review, American Economic Association, vol. 74(5), pages 986-994, December.
    36. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    37. Gallup, John & Sachs, Jeffrey, 1999. "Geography and Economic Development," Harvard Institute for International Development (HIID) Papers 294434, Harvard University, Kennedy School of Government.
    38. Claudius Gräbner & Birte Strunk, 2020. "Pluralism in economics: its critiques and their lessons," Journal of Economic Methodology, Taylor & Francis Journals, vol. 27(4), pages 311-329, October.
    39. Joshua Angrist & Alan Krueger, 2001. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments," Working Papers 834, Princeton University, Department of Economics, Industrial Relations Section..
    40. Dan A. Black & Seth G. Sanders & Evan J. Taylor & Lowell J. Taylor, 2015. "The Impact of the Great Migration on Mortality of African Americans: Evidence from the Deep South," American Economic Review, American Economic Association, vol. 105(2), pages 477-503, February.
    41. repec:osf:socarx:5bck9_v1 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Braunfels, Elias, 2016. "Further Unbundling Institutions," Discussion Paper Series in Economics 13/2016, Norwegian School of Economics, Department of Economics.
    2. Bennett, Daniel L. & Faria, Hugo J. & Gwartney, James D. & Morales, Daniel R., 2017. "Economic Institutions and Comparative Economic Development: A Post-Colonial Perspective," World Development, Elsevier, vol. 96(C), pages 503-519.
    3. Jubril Animashaun & Ada Wossink & Katsushi S. Imai, 2023. "Colonialism, Institutional Quality, and the Resource Curse," Discussion Paper Series DP2023-19, Research Institute for Economics & Business Administration, Kobe University.
    4. Valentina A Assenova & Matthew Regele, 2017. "Revisiting the effect of colonial institutions on comparative economic development," PLOS ONE, Public Library of Science, vol. 12(5), pages 1-16, May.
    5. Stimpfle, Alexander & Stadelmann, David, 2015. "The Impact of Fundamental Development Factors on Different Income Groups: International Evidence," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113128, Verein für Socialpolitik / German Economic Association.
    6. Tang, Sam Hak Kan & Groenewold, Nicolaas & Leung, Charles Ka Yui, 2008. "The link between institutions, technical change and macroeconomic volatility," Journal of Macroeconomics, Elsevier, vol. 30(4), pages 1520-1549, December.
    7. Natkhov, T. & Polishchuk, L., 2017. "Political Economy of Institutions and Development: The Importance of Being Inclusive. Reflection on "Why Nations Fail" by D. Acemoglu and J. Robinson. Part I. Institutions and Economic Devel," Journal of the New Economic Association, New Economic Association, vol. 34(2), pages 12-38.
    8. Kyriacou, Andreas P., 2016. "Individualism–collectivism, governance and economic development," European Journal of Political Economy, Elsevier, vol. 42(C), pages 91-104.
    9. Ho, Chi Pui, 2016. "GeoPopulation-Institution Hypothesis: Reconciling American Development Process and Reversal of Fortune within a Unified Growth Framework," MPRA Paper 73863, University Library of Munich, Germany.
    10. Mauricio Cárdenas, 2007. "Economic Growth in Colombia : a reversal of "fortune"?," Revista ESPE - Ensayos sobre Política Económica, Banco de la Republica de Colombia, vol. 25(53), pages 220-259, January.
    11. Richard S.J. Tol, 2025. "Climate Determinism Reborn," Working Paper Series 0725, Department of Economics, University of Sussex Business School.
    12. Messono, Omang & Asongu, Simplice, 2021. "Historical Prevalence of Infectious Diseases and Entrepreneurship: the Role of Institutions in 125 Countries," MPRA Paper 111842, University Library of Munich, Germany.
    13. P. Dorian Owen, 2017. "Evaluating Ingenious Instruments for Fundamental Determinants of Long-Run Economic Growth and Development," Econometrics, MDPI, vol. 5(3), pages 1-33, September.
    14. Fuchs-Schündeln, N. & Hassan, T.A., 2016. "Natural Experiments in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 923-1012, Elsevier.
    15. James Feyrer, 2019. "Trade and Income—Exploiting Time Series in Geography," American Economic Journal: Applied Economics, American Economic Association, vol. 11(4), pages 1-35, October.
    16. Erkan Goeren, 2017. "The Role of Novelty-Seeking Traits in Contemporary Knowledge Creation," Working Papers V-402-17, University of Oldenburg, Department of Economics, revised Sep 2017.
    17. Ying Fang & Yang Zhao, 2013. "Do Institutions Matter? Estimating the Effect of Institutions on Econo- mic Performance in China," Working Papers 2013-10-14, Wang Yanan Institute for Studies in Economics (WISE), Xiamen University.
    18. Matthew H Bonds & Andrew P Dobson & Donald C Keenan, 2012. "Disease Ecology, Biodiversity, and the Latitudinal Gradient in Income," PLOS Biology, Public Library of Science, vol. 10(12), pages 1-12, December.
    19. Pham, Hien Thuc & Carmignani, Fabrizio & Kler, Parvinder, 2018. "Thrift culture and the size of government," Economic Modelling, Elsevier, vol. 70(C), pages 571-578.
    20. Martina Cioni & Giovanni Federico & Michelangelo Vasta, 2023. "Is economic history changing its nature? Evidence from top journals," Cliometrica, Springer;Cliometric Society (Association Francaise de Cliométrie), vol. 17(1), pages 23-48, January.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:rwirep:330180. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/rwiesde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.