IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this paper

Teaching Replication in Quantitative Empirical Economics

Listed author(s):
  • Jan H. Höffler

    ()

In empirical economics, a twofold lack of incentives leads to chronic problems with replicability: For authors of empirical studies providing replicable material is not awarded in the same way as publishing new irreplicable studies is. Neither is authoring replication studies. We offer a strategy to set incentives for replicability and replication. By integrating replication studies in the education of young scholars, we raise the awareness for the importance of replicability among the next generation of researchers and ensure that a big number of scientists get incentives to write replication studies: credit points and the prospect of publications at least of working papers already during their time as students. By raising the number of researchers involved in replication and by providing an infrastructure for sharing their information, on the one hand we help to lower the amount of work researchers need to put into making their studies replicable. On the other hand, we facilitate the dissemination of insights derived from replication studies. This as a side effect imposes a significant threat of detection of irreplicable research, following the cases of recently introduced wiki projects for the revelation of plagiarism. In contrast to previous efforts like the report on the American Economic Review Data Availability Compliance Project, with our project we build the basis for the first replicable review paper on reblicability as we give account of which studies were tested and which results were found in each case. After exploring several dozen studies published in highly ranked journals, we have not yet determined a single case where we see replicability is fully ensured. We identified two main problems: First, not all published results can be obtained from the replication material provided. Second, information about how the used data were obtained from the raw data is hardly ever sufficient. For our investigation, we gave seminars at several faculties. We set up a wiki project for documenting the results of our replications as well as those found in the literature. In our database, we provide information about more than 1800 empirical studies, especially with regards to the availability of material for their replication. We invite for discussion to develop standards for how to make research replicable and how to write replication studies. For this we provide information about existing projects that facilitate the sharing of material for empirical econometric research.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://curriculumconference2013.weaconferences.net/wp-content/uploads/sites/5/2013/05/WEA-CurriculumConference2013-HOFFLER.pdf
File Function: First publicly available version, World Economics Association conference, 2013
Download Restriction: no

File URL: http://www.touteconomie.org/afse2014/index.php/meeting2014/lyon/paper/viewFile/348/183
File Function: Second, revised version, French Economic Association conference 2014
Download Restriction: no

File URL: http://www.eea-esem.com/files/papers/eea-esem/2014/3108/Teaching%20Replication%20in%20Quantitative%20Empirical%20Economics%20-%20paper%20for%20presentation%20at%20the%20Econometric%20Society%20European%20Meeting%202014.pdf
File Function: Third, revised version, Econometric Society European Meeting 2014
Download Restriction: no

File URL: https://editorialexpress.com/cgi-bin/conference/download.cgi?db_name=NOEG2015&paper_id=62
File Function: Fourth, revised version, Austrian Economic Association conference 2015
Download Restriction: no

File URL: https://www.aeaweb.org/conference/2016/retrieve.php?pdfid=1400
File Function: Fifth, revised version, American Economic Association Annual Meeting 2016
Download Restriction: no

Paper provided by Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project in its series Replication Working Papers with number 2/2014.

as
in new window

Length: 13 pages
Date of creation: Mar 2014
Handle: RePEc:stg:wpaper:2014_02
Contact details of provider: Postal:
Humboldtallee 3, 37073 Göttingen

Phone: 0551 - 39 27282
Fax: 0551 - 39 27279
Web page: http://replication.uni-goettingen.de

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as
in new window


  1. Denis Huschka & Gert G. Wagner, 2012. "Data accessibility is not sufficient for making replication studies a matter of course," Working Paper Series of the German Council for Social and Economic Data 195, German Council for Social and Economic Data (RatSWD).
  2. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
  3. Robert Whaples, 2006. "The Costs of Critical Commentary in Economics Journals," Econ Journal Watch, Econ Journal Watch, vol. 3(2), pages 275-282, May.
  4. Paul Wohlfarth, 2012. "Replication in the narrow sense of Banzhaf/Walsh (2008)," Replication Working Papers 2/2012, Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project.
  5. McCullough, B. D. & McGeary, Kerry Anne & Harrison, Teresa D., 2006. "Lessons from the JMCB Archive," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(4), pages 1093-1107, June.
  6. Björn Zakula, 2012. "Narrow Replication of Ashcraft (2005): Are Banks Really Special?," Replication Working Papers 1/2012, Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project.
  7. Christoph Weißer, 2014. "Replication in the narrow sense of "Financial Stability, the Trilemma, and International Reserves" (Obstfeld, Shambaugh & Taylor 2010)," Replication Working Papers 1/2014, Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project.
  8. Philip R. P. Coelho & James E. McClure, 2006. "Why Has Critical Commentary Been Curtailed at Top Economics Journals? A Reply to Robert Whaples," Econ Journal Watch, Econ Journal Watch, vol. 3(2), pages 283-291, May.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:stg:wpaper:2014_02. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Jan H. Höffler)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.