IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-01939942.html
   My bibliography  Save this paper

How to make a pie? Reproducible Research for Empirical Economics & Econometrics

Author

Listed:
  • Valérie Orozco

    (TSE - Toulouse School of Economics - UT1 - Université Toulouse 1 Capitole - Université Fédérale Toulouse Midi-Pyrénées - INRA - Institut National de la Recherche Agronomique - EHESS - École des hautes études en sciences sociales - CNRS - Centre National de la Recherche Scientifique)

  • Christophe Bontemps

    (TSE - Toulouse School of Economics - UT1 - Université Toulouse 1 Capitole - Université Fédérale Toulouse Midi-Pyrénées - INRA - Institut National de la Recherche Agronomique - EHESS - École des hautes études en sciences sociales - CNRS - Centre National de la Recherche Scientifique)

  • Elise Maigne

    (US ODR - Observatoire des Programmes Communautaires de Développement Rural - INRA - Institut National de la Recherche Agronomique)

  • Virginie Piguet

    (CESAER - Centre d'Economie et de Sociologie Rurales Appliquées à l'Agriculture et aux Espaces Ruraux - INRA - Institut National de la Recherche Agronomique - AgroSup Dijon - Institut National Supérieur des Sciences Agronomiques, de l'Alimentation et de l'Environnement)

  • Annie Hofstetter

    (LAMETA - Laboratoire Montpelliérain d'Économie Théorique et Appliquée - UM1 - Université Montpellier 1 - UPVM - Université Paul-Valéry - Montpellier 3 - INRA - Institut National de la Recherche Agronomique - Montpellier SupAgro - Centre international d'études supérieures en sciences agronomiques - UM - Université de Montpellier - CNRS - Centre National de la Recherche Scientifique - Montpellier SupAgro - Institut national d’études supérieures agronomiques de Montpellier - Institut Agro - Institut national d'enseignement supérieur pour l'agriculture, l'alimentation et l'environnement)

  • Anne Marie Lacroix

    (GAEL - Laboratoire d'Economie Appliquée de Grenoble - UGA [2016-2019] - Université Grenoble Alpes [2016-2019] - INRA - Institut National de la Recherche Agronomique - CNRS - Centre National de la Recherche Scientifique - Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology)

  • Fabrice Levert

    (SMART-LERECO - Structures et Marché Agricoles, Ressources et Territoires - INRA - Institut National de la Recherche Agronomique - AGROCAMPUS OUEST - Institut Agro - Institut national d'enseignement supérieur pour l'agriculture, l'alimentation et l'environnement)

  • Jean-Marc Rousselle

    (LAMETA - Laboratoire Montpelliérain d'Économie Théorique et Appliquée - UM1 - Université Montpellier 1 - UPVM - Université Paul-Valéry - Montpellier 3 - INRA - Institut National de la Recherche Agronomique - Montpellier SupAgro - Centre international d'études supérieures en sciences agronomiques - UM - Université de Montpellier - CNRS - Centre National de la Recherche Scientifique - Montpellier SupAgro - Institut national d’études supérieures agronomiques de Montpellier - Institut Agro - Institut national d'enseignement supérieur pour l'agriculture, l'alimentation et l'environnement)

Abstract

Les recherches empiriques en sciences humaines et sociales nécessitent la manipulation de nombreux fichiers : différents jeux de données, de multiples programmes, qu'ils soient destinés à la manipulation de données, aux traitements statistiques, aux estimations économétriques ou à des simulations, et de nombreux fichiers successifs de résultats. Maitriser l'ensemble des étapes du projet de recherche est indispensable si l'on souhaite pouvoir reproduire ou répliquer les résultats à long terme. Cette rigueur (fiabilité, traçabilité, reproductibilité), est désormais de plus en plus exigée par notre profession ainsi que par les éditeurs de revues scientifiques. Après avoir mis en évidence les enjeux de cette « recherche reproductible » (Claerbout, 1990) pour les économistes, nous en déclinons les principes autour de trois axes simples : organiser les étapes du projet et les fichiers, écrire des programmes clairs et documentés, et automatiser le plus possible les opérations jusqu'au document présentant les résultats. Ces principes sont alors illustrés par différentes bonnes pratiques, en allant des plus simples au plus sophistiquées, avec un focus particulier sur les fonctionnalités des logiciels les plus courants en économie (Stata, R, SAS, Matlab, Mathematica, Gams).

Suggested Citation

  • Valérie Orozco & Christophe Bontemps & Elise Maigne & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2017. "How to make a pie? Reproducible Research for Empirical Economics & Econometrics," Post-Print hal-01939942, HAL.
  • Handle: RePEc:hal:journl:hal-01939942
    Note: View the original document on HAL open archive server: https://hal.archives-ouvertes.fr/hal-01939942
    as

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ben Jann, 2017. "Creating HTML or Markdown documents from within Stata using webdoc," Stata Journal, StataCorp LP, vol. 17(1), pages 3-38, March.
    2. Thomas Herndon & Michael Ash & Robert Pollin, 2014. "Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff," Cambridge Journal of Economics, Oxford University Press, vol. 38(2), pages 257-279.
    3. Andrew C. Chang & Phillip Li, 2017. "A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time," American Economic Review, American Economic Association, vol. 107(5), pages 60-64, May.
    4. Jan H. Höffler, 2014. "Teaching Replication in Quantitative Empirical Economics," Replication Working Papers 2/2014, Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project.
    5. Björk, Bo-Christer & Solomon, David, 2013. "The publishing delay in scholarly peer-reviewed journals," Journal of Informetrics, Elsevier, vol. 7(4), pages 914-923.
    6. Jan H. Höffler, 2017. "Replication and Economics Journal Policies," American Economic Review, American Economic Association, vol. 107(5), pages 52-55, May.
    7. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    8. Pascaline Dupas & Jonathan Robinson, 2013. "Savings Constraints and Microenterprise Development: Evidence from a Field Experiment in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 5(1), pages 163-192, January.
    9. Carmen M. Reinhart & Kenneth S. Rogoff, 2010. "Growth in a Time of Debt," American Economic Review, American Economic Association, vol. 100(2), pages 573-578, May.
    10. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 40(3), pages 715-733, August.
    11. E. F. Haghish, 2016. "markdoc: Literate programming in Stata," Stata Journal, StataCorp LP, vol. 16(4), pages 964-988, December.
    12. Lenth, Russell V. & Højsgaard, Søren, 2007. "SASWeave: Literate Programming Using SAS," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 19(i08).
    13. Denis Huschka, 2013. "Why should we share our data, how can it be organized, and what are the challenges ahead?," RatSWD Working Papers 216, German Data Forum (RatSWD).
    14. E. F. Haghish, 2016. "Rethinking literate programming in statistics," Stata Journal, StataCorp LP, vol. 16(4), pages 938-963, December.
    15. Daniel S. Hamermesh, 2013. "Six Decades of Top Economics Publishing: Who and How?," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 162-172, March.
    16. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    17. Ties de Kok, 2016. "Combine Stata with Python using the Jupyter Notebook," 2016 Stata Conference 2, Stata Users Group.
    18. Jeffrey Racine, 2017. "Energy, Economics & Replication," Department of Economics Working Papers 2017-02, McMaster University.
    19. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    20. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    21. Vlaeminck, Sven & Herrmann, Lisa-Kristin, 2015. "Data Policies and Data Archives: A New Paradigm for Academic Publishing in Economic Sciences?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 145-155.
    22. B.D. McCullough, 2009. "Open Access Economics Journals and the Market for Reproducible Economic Research," Economic Analysis and Policy, Elsevier, vol. 39(1), pages 117-126, March.
    23. Roseline Bilina & Steve Lawford, 2012. "Python for Unified Research in Econometrics and Statistics," Econometric Reviews, Taylor & Francis Journals, vol. 31(5), pages 558-591, September.
    24. McCullough, B. D. & McGeary, Kerry Anne & Harrison, Teresa D., 2006. "Lessons from the JMCB Archive," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(4), pages 1093-1107, June.
    25. Barreto,Humberto & Howland,Frank, 2006. "Introductory Econometrics," Cambridge Books, Cambridge University Press, number 9780521843195, October.
    26. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1988. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project: Reply," American Economic Review, American Economic Association, vol. 78(5), pages 1162-1163, December.
    27. Ben Jann, 2016. "Creating LaTeX documents from within Stata using texdoc," Stata Journal, StataCorp LP, vol. 16(2), pages 245-263, June.
    28. Sebastian Galiani & Paul Gertler & Mauricio Romero, 2017. "Incentives for Replication in Economics," NBER Working Papers 23576, National Bureau of Economic Research, Inc.
    29. Russell Lenth & Søren Højsgaard, 2011. "Reproducible statistical analysis with multiple languages," Computational Statistics, Springer, vol. 26(3), pages 419-426, September.
    30. Germán Rodríguez, 2017. "Literate data analysis with Stata and Markdown," Stata Journal, StataCorp LP, vol. 17(3), pages 600-618, September.
    31. Ben Jann, 2016. "texdoc 2.0: An update on creating LaTeX documents from within Stata," United Kingdom Stata Users' Group Meetings 2016 04, Stata Users Group.
    32. Schulte, Eric & Davison, Dan & Dye, Thomas & Dominik, Carsten, 2012. "A Multi-Language Computing Environment for Literate Programming and Reproducible Research," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 46(i03).
    33. Roger Koenker & Achim Zeileis, 2009. "On reproducible econometric research," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(5), pages 833-847.
    34. Evan Meredith & Jeffrey S. Racine, 2009. "Towards reproducible econometric research: the Sweave framework," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(2), pages 366-374, March.
    35. J. Scott Long, 2009. "The Workflow of Data Analysis Using Stata," Stata Press books, StataCorp LP, number wdaus, March.
    36. Hunter, John E, 2001. "The Desperate Need for Replications," Journal of Consumer Research, Oxford University Press, vol. 28(1), pages 149-158, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 62-83.
    2. Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre (Seville site).
    3. Hernández Alemán, Anastasia & León, Carmelo J., 2018. "La Réplica en el Análisis Económico Aplicado/Replication in Applied Economic Analysis," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 36, pages 317-332, Enero.
    4. Christophe Hurlin & Christophe Pérignon, 2020. "Reproducibility Certification in Economics Research," Working Papers hal-02896404, HAL.
    5. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    6. Nicolas Vallois & Dorian Jullien, 2017. "Replication in experimental economics: A historical and quantitative approach focused on public good game experiments," Working Papers halshs-01651080, HAL.
    7. Nicolas Vallois & Dorian Jullien, 2017. "Replication in experimental economics: A historical and quantitative approach focused on public good game experiments," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01651080, HAL.
    8. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    9. Daniels, Gerald Eric & Kakar, Venoo, 2018. "Normalized CES supply-side system approach: How to replicate Klump, McAdam, and Willman (2007)," Economics - The Open-Access, Open-Assessment E-Journal, Kiel Institute for the World Economy (IfW), vol. 12, pages 1-11.
    10. Nicolas Vallois & Dorian Jullien, 2017. "Replication in Experimental Economics: A Historical and Quantitative Approach Focused on Public Good Game Experiments," GREDEG Working Papers 2017-21, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    11. Andreoli-Versbach, Patrick & Mueller-Langer, Frank, 2014. "Open access to data: An ideal professed but not practised," Research Policy, Elsevier, vol. 43(9), pages 1621-1633.
    12. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    13. Christensen, Garret & Miguel, Edward & Sturdy, Jennifer, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," MetaArXiv 9a3rw, Center for Open Science.
    14. Daniels, Gerald Eric & Kakar, Venoo, 2017. "Normalized CES supply-side system approach: How to replicate Klump, McAdam, and Willman (Review of Economics and Statistics, 2007)," Economics Discussion Papers 2017-70, Kiel Institute for the World Economy (IfW).
    15. Horton, Joanne & Krishna Kumar, Dhanya & Wood, Anthony, 2020. "Detecting academic fraud using Benford law: The case of Professor James Hunton," Research Policy, Elsevier, vol. 49(8).
    16. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    17. Vlaeminck, Sven & Herrmann, Lisa-Kristin, 2015. "Data Policies and Data Archives: A New Paradigm for Academic Publishing in Economic Sciences?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 145-155.
    18. Vlaeminck, Sven & Podkrajac, Felix, 2017. "Journals in Economic Sciences: Paying Lip Service to Reproducible Research?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 1-16.
    19. Vlaeminck, Sven, 2013. "Data Management in Scholarly Journals and Possible Roles for Libraries - Some Insights from EDaWaX," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 49-79.
    20. Joachim Wagner, 2016. "From Estimation Results to Stylized Facts: Twelve Recommendations for Empirical Research in International Activities of Heterogeneous Firms," World Scientific Book Chapters, in: Microeconometrics of International Trade, chapter 15, pages 479-514, World Scientific Publishing Co. Pte. Ltd..

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-01939942. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://hal.archives-ouvertes.fr/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.