Reproducibility of Empirical Results: Evidence from 1,000 Tests in Finance
Author
Abstract
Suggested Citation
DOI: 10.2139/ssrn.4064172
Download full text from publisher
To our knowledge, this item is not available for download. To find whether it is available, there are three options:1. Check below whether another version of this item is available online.
2. Check on the provider's web page whether it is in fact available.
3. Perform a search for a similarly titled item that would be available.
References listed on IDEAS
- Garret Christensen & Edward Miguel, 2018.
"Transparency, Reproducibility, and the Credibility of Economics Research,"
Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
- Garret S. Christensen & Edward Miguel, 2016. "Transparency, Reproducibility, and the Credibility of Economics Research," NBER Working Papers 22989, National Bureau of Economic Research, Inc.
- Christensen, Garret & Miguel, Edward & Sturdy, Jennifer, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," MetaArXiv 9a3rw, Center for Open Science.
- Christensen, Garret & Miguel, Edward, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," Department of Economics, Working Paper Series qt52h6x1cq, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
- Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017.
"What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?,"
American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
- Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2016. "What is Meant by ‘Replication’ and Why Does It Encounter Resistance in Economics?," Working Papers in Economics 16/34, University of Canterbury, Department of Economics and Finance.
- Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
- B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
- Kewei Hou & Chen Xue & Lu Zhang, 2020. "Replicating Anomalies," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2019-2133.
- Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021.
"The influence of hidden researcher decisions in applied microeconomics,"
Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
- Huntington-Klein, Nick & Arenas, Andreu & Beam, Emily A. & Bertoni, Marco & Bloem, Jeffrey R. & Burli, Pralhad & Chen, Naibin & Greico, Paul & Ekpe, Godwin & Pugatch, Todd & Saavedra, Martin & Stopnit, 2020. "The Influence of Hidden Researcher Decisions in Applied Microeconomics," IZA Discussion Papers 13233, Institute of Labor Economics (IZA).
- Huntington-Klein, Nick & Arenas, Andreu & Beam, Emily & Bertoni, Marco & Bloem, Jeffrey R. & Burli, Pralhad & Chen, Naibin & Greico, Paul & Ekpe, Godwin & Pugatch, Todd & Saavedra, Martin & Stopnitzky, 2020. "The Influence of Hidden Researcher Decisions in Applied Microeconomics," GLO Discussion Paper Series 537, Global Labor Organization (GLO).
- McCullough, B. D. & McGeary, Kerry Anne & Harrison, Teresa D., 2006. "Lessons from the JMCB Archive," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(4), pages 1093-1107, June.
- Edward Miguel, 2021.
"Evidence on Research Transparency in Economics,"
Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 193-214, Summer.
- Miguel, Edward, 2021. "Evidence on Research Transparency in Economics," Department of Economics, Working Paper Series qt7fc7s8cd, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
- Grossmann, Axel & Lee, Allissa, 2022. "An analysis of finance journal accessibility: Author inclusivity and journal quality," Journal of Banking & Finance, Elsevier, vol. 138(C).
- Andrew C. Chang & Phillip Li, 2017. "A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time," American Economic Review, American Economic Association, vol. 107(5), pages 60-64, May.
- Paul Gertler & Sebastian Galiani & Mauricio Romero, 2018. "How to make replication the norm," Nature, Nature, vol. 554(7693), pages 417-419, February.
- Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
- Rotem Botvinik-Nezer & Felix Holzmeister & Colin F. Camerer & Anna Dreber & Juergen Huber & Magnus Johannesson & Michael Kirchler & Roni Iwanir & Jeanette A. Mumford & R. Alison Adcock & Paolo Avesani, 2020. "Variability in the analysis of a single neuroimaging dataset by many teams," Nature, Nature, vol. 582(7810), pages 84-88, June.
- Sylvérie Herbert & Hautahi Kingi & Flavio Stanchi & Lars Vilhubern, 2021. "The Reproducibility of Economics Research: A Case Study," Working papers 853, Banque de France.
- Theis Ingerslev Jensen & Bryan T. Kelly & Lasse Heje Pedersen, 2021. "Is There A Replication Crisis In Finance?," NBER Working Papers 28432, National Bureau of Economic Research, Inc.
- Esther Duflo & Hilary Hoynes, 2018. "Report of the Search Committee to Appoint a Data Editor for the AEA," AEA Papers and Proceedings, American Economic Association, vol. 108, pages 745-745, May.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Dreber, Anna & Johannesson, Magnus, 2023.
"A framework for evaluating reproducibility and replicability in economics,"
Ruhr Economic Papers
1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," I4R Discussion Paper Series 38, The Institute for Replication (I4R).
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Dreber, Anna & Johannesson, Magnus, 2023.
"A framework for evaluating reproducibility and replicability in economics,"
I4R Discussion Paper Series
38, The Institute for Replication (I4R).
- Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
- Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021.
"The influence of hidden researcher decisions in applied microeconomics,"
Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
- Huntington-Klein, Nick & Arenas, Andreu & Beam, Emily & Bertoni, Marco & Bloem, Jeffrey R. & Burli, Pralhad & Chen, Naibin & Greico, Paul & Ekpe, Godwin & Pugatch, Todd & Saavedra, Martin & Stopnitzky, 2020. "The Influence of Hidden Researcher Decisions in Applied Microeconomics," GLO Discussion Paper Series 537, Global Labor Organization (GLO).
- Huntington-Klein, Nick & Arenas, Andreu & Beam, Emily A. & Bertoni, Marco & Bloem, Jeffrey R. & Burli, Pralhad & Chen, Naibin & Greico, Paul & Ekpe, Godwin & Pugatch, Todd & Saavedra, Martin & Stopnit, 2020. "The Influence of Hidden Researcher Decisions in Applied Microeconomics," IZA Discussion Papers 13233, Institute of Labor Economics (IZA).
- Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023.
"Do economists replicate?,"
Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
- Fiala, Nathan & Neubauer, Florian & Peters, Jörg, 2022. "Do economists replicate?," Ruhr Economic Papers 939, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do Economists Replicate?," I4R Discussion Paper Series 13, The Institute for Replication (I4R).
- Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021.
"Non-Standard Errors,"
Documents de travail du Centre d'Economie de la Sorbonne
21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian Brownlees & Javier Gil-Bazo, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-DÃaz & Menachem Abudy & To, 2021. "Non-Standard Errors," Working Paper Series, Social and Economic Sciences 2021-11, Faculty of Social and Economic Sciences, Karl-Franzens-University Graz.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
- Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
- Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
- Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
- Moinas, Sophie & Declerck, Fany & Menkveld, Albert J. & Dreber, Anna, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
- Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Díaz & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
- Ferrara, Gerardo & Jurkatis, Simon, 2021. "Non-standard errors," Bank of England working papers 955, Bank of England.
- Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
- Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
- Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Janeway Institute Working Papers 2112, Faculty of Economics, University of Cambridge.
- Wolff, Christian & Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, C.E.P.R. Discussion Papers.
- Valérie Orozco & Christophe Bontemps & Élise Maigné & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2017.
"How to make a pie? Reproducible Research for Empirical Economics & Econometrics,"
Post-Print
hal-01939942, HAL.
- Orozco, Valérie & Bontemps, Christophe & Maigné, Elise & Piguet, V. & Hofstetter, A. & Lacroix, Anne & Levert, F. & Rousselle, J.M, 2018. "How To Make A Pie: Reproducible Research for Empirical Economics & Econometrics," TSE Working Papers 18-933, Toulouse School of Economics (TSE).
- Valérie Orozco & Christophe Bontemps & Élise Maigné & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2019. "How To Make A Pie: Reproducible Research for Empirical Economics & Econometrics," Working Papers hal-02025843, HAL.
- Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023.
"Reproducibility in Management Science,"
Department for Strategy and Innovation Working Paper Series
03/2023, WU Vienna University of Economics and Business.
- Miloš Fišar & Ben Greiner & Christoph Huber & Elena Katok & Ali I Ozkes & The Management Science Reproducibility Collaboration, 2024. "Reproducibility in Management Science," Post-Print hal-04370984, HAL.
- Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Collaboration, Management Science Reproducibility, 2023. "Reproducibility in Management Science," OSF Preprints mydzv, Center for Open Science.
- Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018.
"How to run an experimental auction: A review of recent advances,"
Working Papers
2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
- Canavari, Maurizio & Drichoutis, Andreas C. & Lusk, Jayson L. & Nayga, Rodolfo, 2018. "How to run an experimental auction: A review of recent advances," MPRA Paper 89715, University Library of Munich, Germany.
- Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019.
"The Dozen Things Experimental Economists Should Do (More of),"
Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
- Eszter Czibor & David Jimenez-Gomez & John List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Artefactual Field Experiments 00648, The Field Experiments Website.
- Eszter Czibor & David Jimenez-Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," NBER Working Papers 25451, National Bureau of Economic Research, Inc.
- Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
- Hernández Alemán, Anastasia & León, Carmelo J., 2018. "La Réplica en el Análisis Económico Aplicado/Replication in Applied Economic Analysis," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 36, pages 317-332, Enero.
- Christophe Hurlin & Christophe Pérignon, 2020. "Reproducibility Certification in Economics Research," Working Papers hal-02896404, HAL.
- Sylvérie Herbert & Hautahi Kingi & Flavio Stanchi & Lars Vilhubern, 2021. "The Reproducibility of Economics Research: A Case Study," Working papers 853, Banque de France.
- Valérie Orozco & Christophe Bontemps & Elise Maigné & Virginie Piguet & Annie Hofstetter & Anne Lacroix & Fabrice Levert & Jean‐Marc Rousselle, 2020.
"How To Make A Pie: Reproducible Research For Empirical Economics And Econometrics,"
Journal of Economic Surveys, Wiley Blackwell, vol. 34(5), pages 1134-1169, December.
- Valérie Orozco & Christophe Bontemps & Élise Maigné & Virginie Piguet & Annie Hofstetter & Anne Lacroix & Fabrice Levert & Jean‐marc Rousselle, 2020. "How to make a pie: Reproductible research for empirical economics and econometrics," Post-Print hal-03014999, HAL.
- Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre.
- Mueller-Langer, Frank & Andreoli-Versbach, Patrick, 2018.
"Open access to research data: Strategic delay and the ambiguous welfare effects of mandatory data disclosure,"
Information Economics and Policy, Elsevier, vol. 42(C), pages 20-34.
- Frank Mueller-Langer & Patrick Andreoli-Versbach, 2014. "Open Access to Research Data: Strategic Delay and the Ambiguous Welfare Effects of Mandatory Data Disclosure," RatSWD Working Papers 239, German Data Forum (RatSWD).
- Mueller-Langer, Frank & Andreoli-Versbach, Patrick, 2014. "Open Access to Research Data: Strategic Delay and the Ambiguous Welfare Effects of Mandatory Data Disclosure," Discussion Papers in Economics 21037, University of Munich, Department of Economics.
- Müller-Langer, Frank & Andreoli-Versbach, Patrick, 2018. "Open access to research data: Strategic delay and the ambiguous welfare effects of mandatory data disclosure," Munich Reprints in Economics 62831, University of Munich, Department of Economics.
- Graham Elliott & Nikolay Kudrin & Kaspar Wüthrich, 2022.
"Detecting p‐Hacking,"
Econometrica, Econometric Society, vol. 90(2), pages 887-906, March.
- Graham Elliott & Nikolay Kudrin & Kaspar Wuthrich, 2019. "Detecting p-hacking," Papers 1906.06711, arXiv.org, revised May 2021.
- Elliott, Graham & Kudrin, Nikolay & Wüthrich, Kaspar, 2022. "Detecting p‐Hacking," University of California at San Diego, Economics Working Paper Series qt2p04s3dr, Department of Economics, UC San Diego.
- Bull, Charles & Courty, Pascal & Doyon, Maurice & Rondeau, Daniel, 2019.
"Failure of the Becker–DeGroot–Marschak mechanism in inexperienced subjects: New tests of the game form misconception hypothesis,"
Journal of Economic Behavior & Organization, Elsevier, vol. 159(C), pages 235-253.
- Courty, Pascal & Bull, Charlie & Rondeau, Daniel & Doyon, Maurice, 2017. "Failure of the Becker-Degroot-Marschak Mechanism in Inexperienced Subjects: New Tests of the Game Form Misconception Hypothesis," CEPR Discussion Papers 12479, C.E.P.R. Discussion Papers.
- Joakim A. Weill & Matthieu Stigler & Olivier Deschenes & Michael R. Springborn, 2021. "Researchers' Degrees-of-Flexibility and the Credibility of Difference-in-Differences Estimates: Evidence From the Pandemic Policy Evaluations," NBER Working Papers 29550, National Bureau of Economic Research, Inc.
- Nicolas Vallois & Dorian Jullien, 2017. "Replication in experimental economics: A historical and quantitative approach focused on public good game experiments," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01651080, HAL.
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:wpaper:hal-03810013. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.