IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_7942.html
   My bibliography  Save this paper

A Journal-Based Replication of "Being Chosen to Lead"

Author

Listed:
  • Allan Drazen
  • Anna Dreber
  • Erkut Y. Ozbay
  • Erik Snowberg

Abstract

Recent large-scale replications of social science experiments provide important information on the reliability of experimental research. Unfortunately, there exist no mechanisms to ensure replications are done. We propose such a mechanism: journal-based replication, in which the publishing journal contracts for a replication between acceptance and publication. We discuss what we learned from a proof-of-concept journal-based replication at the Journal of Public Economics. Our experience indicates that journal-based replication would be relatively straightforward to implement for laboratory experiments.

Suggested Citation

  • Allan Drazen & Anna Dreber & Erkut Y. Ozbay & Erik Snowberg, 2019. "A Journal-Based Replication of "Being Chosen to Lead"," CESifo Working Paper Series 7942, CESifo.
    • Handle: RePEc:ces:ceswps:_7942
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp7942.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    2. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    3. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1986. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project," American Economic Review, American Economic Association, vol. 76(4), pages 587-603, September.
    4. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    5. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    6. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    7. Ron Shachar & Barry Nalebuff, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Comment," American Economic Review, American Economic Association, vol. 94(1), pages 382-390, March.
    8. Luigi Butera & John A. List, 2017. "An Economic Approach to Alleviate the Crises of Confidence in Science: With an Application to the Public Goods Game," NBER Working Papers 23335, National Bureau of Economic Research, Inc.
    9. Jesse Chandler & et. al, 2016. "Response to Comment on "Estimating the Reproducibility of Psychological Science"," Mathematica Policy Research Reports cff9c2f16bb544c4bcca530c0, Mathematica Policy Research.
    10. Caroline M. Hoxby, 2005. "Competition Among Public Schools: A Reply to Rothstein (2004)," NBER Working Papers 11216, National Bureau of Economic Research, Inc.
    11. Drazen, Allan & Ozbay, Erkut Y., 2019. "Does “being chosen to lead” induce non-selfish behavior? Experimental evidence on reciprocity," Journal of Public Economics, Elsevier, vol. 174(C), pages 13-21.
    12. Anna, Petrenko, 2016. "Мaркування готової продукції як складова частина інформаційного забезпечення маркетингової діяльності підприємств овочепродуктового підкомплексу," Agricultural and Resource Economics: International Scientific E-Journal, Agricultural and Resource Economics: International Scientific E-Journal, vol. 2(1), March.
    13. Nikolaos Georgantzis & Juan A. Lacomba & Francisco Lagos & Juliette Milgram, 2013. "Trust and reciprocity among Mediterranean countries," Working Papers 2013/09, Economics Department, Universitat Jaume I, Castellón (Spain).
    14. Israel Waichman & Ch’ng Kean Siang & Till Requate & Aric P. Shafran & Eva Camacho-Cuena & Yoshio Iida & Shosh Shahrabani, 2015. "Reciprocity in Labor Market Relationships: Evidence from an Experiment across High-Income OECD Countries," Games, MDPI, vol. 6(4), pages 1-22, October.
    15. Andrew C. Chang & Phillip Li, 2015. "Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"," Finance and Economics Discussion Series 2015-83, Board of Governors of the Federal Reserve System (U.S.).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Luigi Butera & Philip Grossman & Daniel Houser & John List & Marie-Claire Villeval, 2020. "A New Mechanism to Alleviate the Crises of Confidence in Science - With an Application to the Public Goods Game," Artefactual Field Experiments 00684, The Field Experiments Website.
    2. Luigi Butera & Philip J Grossman & Daniel Houser & John A List & Marie Claire Villeval, 2020. "A New Mechanism to Alleviate the Crises of Confidence in Science With An Application to the Public Goods GameA Review," Working Papers halshs-02512932, HAL.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drazen, Allan & Dreber, Anna & Ozbay, Erkut Y. & Snowberg, Erik, 2021. "Journal-based replication of experiments: An application to “Being Chosen to Lead”," Journal of Public Economics, Elsevier, vol. 202(C).
    2. Kewei Hou & Chen Xue & Lu Zhang, 2017. "Replicating Anomalies," NBER Working Papers 23394, National Bureau of Economic Research, Inc.
    3. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    4. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    5. McCullough, B. D., 2018. "Quis custodiet ipsos custodes? Despite evidence to the contrary, the American Economic Review concluded that all was well with its archive," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-13.
    6. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    7. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    8. Hensel, Przemysław G., 2019. "Supporting replication research in management journals: Qualitative analysis of editorials published between 1970 and 2015," European Management Journal, Elsevier, vol. 37(1), pages 45-57.
    9. Patrick Vu, 2022. "Can the Replication Rate Tell Us About Publication Bias?," Papers 2206.15023, arXiv.org, revised Jul 2022.
    10. Lucas C. Coffman & Muriel Niederle & Alistair J. Wilson, 2017. "A Proposal to Organize and Promote Replications," American Economic Review, American Economic Association, vol. 107(5), pages 41-45, May.
    11. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    12. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    13. Qianwei Ying & Tahir Yousaf & Qurat ul Ain & Yasmeen Akhtar & Muhammad Shahid Rasheed, 2019. "Stock Investment and Excess Returns: A Critical Review in the Light of the Efficient Market Hypothesis," JRFM, MDPI, vol. 12(2), pages 1-22, June.
    14. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    15. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 40(3), pages 715-733, August.
    16. Depro, Brooks & Rouse, Kathryn, 2022. "Adapting the case method in an economics capstone research course," International Review of Economics Education, Elsevier, vol. 41(C).
    17. B.D. McCullough & Kerry Anne McGeary & Teresa D. Harrison, 2008. "Do economics journal archives promote replicable research?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 41(4), pages 1406-1420, November.
    18. Denis Fougère & Nicolas Jacquemet, 2019. "Causal Inference and Impact Evaluation," Economie et Statistique / Economics and Statistics, Institut National de la Statistique et des Etudes Economiques (INSEE), issue 510-511-5, pages 181-200.
    19. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    20. Tyler Cowen & Alex Tabarrok, 2016. "A Skeptical View of the National Science Foundation's Role in Economic Research," Journal of Economic Perspectives, American Economic Association, vol. 30(3), pages 235-248, Summer.

    More about this item

    Keywords

    replication; reliability; experiments; journal-based replication;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C92 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Group Behavior

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_7942. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.