A new standard for the analysis and design of replication studies
Author
Abstract
Suggested Citation
DOI: 10.1111/rssa.12493
Download full text from publisher
References listed on IDEAS
- John P A Ioannidis, 2005. "Why Most Published Research Findings Are False," PLOS Medicine, Public Library of Science, vol. 2(8), pages 1-1, August.
- Robbie C M van Aert & Marcel A L M van Assen, 2017. "Bayesian evaluation of effect size after replicating an original study," PLOS ONE, Public Library of Science, vol. 12(4), pages 1-23, April.
- Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018.
"Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015,"
Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
- Camerer, Colin F. & Dreber, Anna & Holzmeister, Felix & Ho, Teck-Hua & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas & Altmejd, Adam & But, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Munich Reprints in Economics 62818, University of Munich, Department of Economics.
- Camerer, Colin & Dreber, Anna & Holzmeister, Felix & Ho, Teck Hua & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," SocArXiv 4hmb6, Center for Open Science.
- David Colquhoun, 2019. "The False Positive Risk: A Proposal Concerning What to Do About p-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 192-201, March.
- Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
- Bayarri M.J. & Mayoral A.M., 2002. "Bayesian Design of," The American Statistician, American Statistical Association, vol. 56, pages 207-214, August.
- Jesse Chandler & et. al, 2016. "Response to Comment on "Estimating the Reproducibility of Psychological Science"," Mathematica Policy Research Reports cff9c2f16bb544c4bcca530c0, Mathematica Policy Research.
- David J. Spiegelhalter & Laurence S. Freedman & Mahesh K. B. Parmar, 1994. "Bayesian Approaches to Randomized Trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 157(3), pages 357-387, May.
Citations
Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
Cited by:
- Freuli, Francesca & Held, Leonhard & Heyard, Rachel, 2022. "Replication Success under Questionable Research Practices - A Simulation Study," I4R Discussion Paper Series 2, The Institute for Replication (I4R).
- Andra-Octavia Roman & Pedro Jimenez-Sandoval & Sebastian Augustin & Caroline Broyart & Ludwig A. Hothorn & Julia Santiago, 2022. "HSL1 and BAM1/2 impact epidermal cell development by sensing distinct signaling peptides," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
- Muradchanian, Jasmine & Hoekstra, Rink & Kiers, Henk & van Ravenzwaaij, Don, 2020. "How Best to Quantify Replication Success? A Simulation Study on the Comparison of Replication Success Metrics," MetaArXiv wvdjf, Center for Open Science.
- Samuel Pawel & Leonhard Held, 2022. "The sceptical Bayes factor for the assessment of replication success," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(3), pages 879-911, July.
- Freuli, Francesca & Held, Leonhard & Heyard, Rachel, 2022. "Replication success under questionable research practices – a simulation study," MetaArXiv s4b65, Center for Open Science.
- Gary L. Rosner & Peter Müller, 2020. "Discussion on “Predictively consistent prior effective sample sizes,” by Beat Neuenschwander, Sebastian Weber, Heinz Schmidli, and Anthony O'Hagan," Biometrics, The International Biometric Society, vol. 76(2), pages 599-601, June.
- Lawrence L. Kupper & Sandra L. Martin, 2022. "Replication study design: confidence intervals and commentary," Statistical Papers, Springer, vol. 63(5), pages 1577-1583, October.
- Leonhard Held, 2020. "The harmonic mean χ2‐test to substantiate scientific findings," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(3), pages 697-708, June.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Muradchanian, Jasmine & Hoekstra, Rink & Kiers, Henk & van Ravenzwaaij, Don, 2020. "How Best to Quantify Replication Success? A Simulation Study on the Comparison of Replication Success Metrics," MetaArXiv wvdjf, Center for Open Science.
- Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018.
"Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015,"
Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
- Camerer, Colin & Dreber, Anna & Holzmeister, Felix & Ho, Teck Hua & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," SocArXiv 4hmb6, Center for Open Science.
- Camerer, Colin F. & Dreber, Anna & Holzmeister, Felix & Ho, Teck-Hua & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas & Altmejd, Adam & But, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Munich Reprints in Economics 62818, University of Munich, Department of Economics.
- Isaiah Andrews & Maximilian Kasy, 2019.
"Identification of and Correction for Publication Bias,"
American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
- Isaiah Andrews & Maximilian Kasy, 2017. "Identification of and correction for publication bias," Papers 1711.10527, arXiv.org.
- Kasy, Maximilian & Andrews, Isaiah, 2018. "Identification of and correction for publication bias," MetaArXiv 49yst, Center for Open Science.
- Isaiah Andrews & Maximilian Kasy, 2017. "Identification of and Correction for Publication Bias," NBER Working Papers 23298, National Bureau of Economic Research, Inc.
- Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019.
"The Dozen Things Experimental Economists Should Do (More of),"
Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
- Eszter Czibor & David Jimenez-Gomez & John List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Artefactual Field Experiments 00648, The Field Experiments Website.
- Eszter Czibor & David Jimenez-Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," NBER Working Papers 25451, National Bureau of Economic Research, Inc.
- Oliver Braganza, 2020. "A simple model suggesting economically rational sample-size choice drives irreproducibility," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-19, March.
- Adam Altmejd & Anna Dreber & Eskil Forsell & Juergen Huber & Taisuke Imai & Magnus Johannesson & Michael Kirchler & Gideon Nave & Colin Camerer, 2019.
"Predicting the replicability of social science lab experiments,"
PLOS ONE, Public Library of Science, vol. 14(12), pages 1-18, December.
- Altmejd, Adam & Dreber, Anna & Forsell, Eskil & Huber, Jürgen & Imai, Taisuke & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Camerer, Colin, 2019. "Predicting the replicability of social science lab experiments," Munich Reprints in Economics 78212, University of Munich, Department of Economics.
- Williams, Cole Randall, 2019. "How redefining statistical significance can worsen the replication crisis," Economics Letters, Elsevier, vol. 181(C), pages 65-69.
- Strømland, Eirik & Torsvik, Gaute, 2019. "Intuitive Prosociality: Heterogeneous Treatment Effects or False Positive?," OSF Preprints hrx2y, Center for Open Science.
- Chin, Jason & Zeiler, Kathryn, 2021. "Replicability in Empirical Legal Research," LawArXiv 2b5k4, Center for Open Science.
- Alexander Frankel & Maximilian Kasy, 2022.
"Which Findings Should Be Published?,"
American Economic Journal: Microeconomics, American Economic Association, vol. 14(1), pages 1-38, February.
- Kasy, Maximilian & Frankel, Alexander, 2018. "Which findings should be published?," MetaArXiv mbvz3, Center for Open Science.
- Stanley, T. D. & Doucouliagos, Chris, 2019. "Practical Significance, Meta-Analysis and the Credibility of Economics," IZA Discussion Papers 12458, Institute of Labor Economics (IZA).
- Brice Corgnet & Cary Deck & Mark DeSantis & Kyle Hampton & Erik O. Kimbrough, 2023.
"When Do Security Markets Aggregate Dispersed Information?,"
Management Science, INFORMS, vol. 69(6), pages 3697-3729, June.
- Brice Corgnet & Cary Deck & Mark Desantis & Kyle Hampton & Erik O. Kimbrough, 2023. "When Do Security Markets Aggregate Dispersed Information?," Post-Print hal-04325683, HAL.
- Bergemann, Dirk & Ottaviani, Marco, 2021.
"Information Markets and Nonmarkets,"
CEPR Discussion Papers
16459, C.E.P.R. Discussion Papers.
- Dirk Bergemann & Marco Ottaviani, 2021. "Information Markets and Nonmarkets," Cowles Foundation Discussion Papers 2296, Cowles Foundation for Research in Economics, Yale University.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad‐Díaz & Menachem (Meni) Abudy , 2024.
"Nonstandard Errors,"
Journal of Finance, American Finance Association, vol. 79(3), pages 2339-2390, June.
- Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021. "Non-Standard Errors," Documents de travail du Centre d'Economie de la Sorbonne 21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian Brownlees & Javier Gil-Bazo, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-DÃaz & Menachem Abudy & To, 2021. "Non-Standard Errors," Working Paper Series, Social and Economic Sciences 2021-11, Faculty of Social and Economic Sciences, Karl-Franzens-University Graz.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
- Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
- Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
- Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
- Moinas, Sophie & Declerck, Fany & Menkveld, Albert J. & Dreber, Anna, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüß, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Menac, 2024. "Nonstandard errors," LSE Research Online Documents on Economics 123002, London School of Economics and Political Science, LSE Library.
- Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
- Ferrara, Gerardo & Jurkatis, Simon, 2021. "Non-standard errors," Bank of England working papers 955, Bank of England.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Díaz & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
- Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
- Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
- Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Janeway Institute Working Papers 2112, Faculty of Economics, University of Cambridge.
- Wolff, Christian & Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, C.E.P.R. Discussion Papers.
- Charles F. Manski, 2018. "Reasonable patient care under uncertainty," Health Economics, John Wiley & Sons, Ltd., vol. 27(10), pages 1397-1421, October.
- Merl, Robert & Stöckl, Thomas & Palan, Stefan, 2023.
"Insider trading regulation and shorting constraints. Evaluating the joint effects of two market interventions,"
Journal of Banking & Finance, Elsevier, vol. 154(C).
- Robert Merl & Thomas Stöckl & Stefan Palan, 2021. "Insider trading regulation and shorting constraints. Evaluating the joint effects of two market interventions," Working Paper Series, Social and Economic Sciences 2021-03, Faculty of Social and Economic Sciences, Karl-Franzens-University Graz.
- Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018.
"How to run an experimental auction: A review of recent advances,"
Working Papers
2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
- Canavari, Maurizio & Drichoutis, Andreas C. & Lusk, Jayson L. & Nayga, Rodolfo, 2018. "How to run an experimental auction: A review of recent advances," MPRA Paper 89715, University Library of Munich, Germany.
- Bull, Charles & Courty, Pascal & Doyon, Maurice & Rondeau, Daniel, 2019.
"Failure of the Becker–DeGroot–Marschak mechanism in inexperienced subjects: New tests of the game form misconception hypothesis,"
Journal of Economic Behavior & Organization, Elsevier, vol. 159(C), pages 235-253.
- Courty, Pascal & Bull, Charlie & Rondeau, Daniel & Doyon, Maurice, 2017. "Failure of the Becker-Degroot-Marschak Mechanism in Inexperienced Subjects: New Tests of the Game Form Misconception Hypothesis," CEPR Discussion Papers 12479, C.E.P.R. Discussion Papers.
- Fernando Hoces de la Guardia & Sean Grant & Edward Miguel, 2021.
"A framework for open policy analysis,"
Science and Public Policy, Oxford University Press, vol. 48(2), pages 154-163.
- Hoces de la Guardia, Fernando & Grant, Sean & Miguel, Edward, 2018. "A Framework for Open Policy Analysis," MetaArXiv jnyqh, Center for Open Science.
- de la Guardia, Fernando Hoces & Grant, Sean & Miguel, Edward, 2021. "A framework for open policy analysis," Department of Economics, Working Paper Series qt05r470xk, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
- Dohmen, Thomas & Quercia, Simone & Willrodt, Jana, 2023.
"A note on salience of own preferences and the consensus effect,"
Journal of Economic Behavior & Organization, Elsevier, vol. 209(C), pages 15-21.
- Thomas Dohmen & Simone Quercia & Jana Willrodt, 2023. "A Note on Salience of Own Preferences and the Consensus Effect," ECONtribute Discussion Papers Series 219, University of Bonn and University of Cologne, Germany.
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:183:y:2020:i:2:p:431-448. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley-Blackwell Digital Licensing or Christopher F. Baum (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.