IDEAS home Printed from https://ideas.repec.org/p/cbt/econwp/18-22.html
   My bibliography  Save this paper

Takeaways from the Special Issue on The Practice of Replication

Author

Abstract

In July 2017, Economics: The Open Access, Open Assessment E-Journal issued a call for papers for a special issue on “The Practice of Replication.” In that call, the journal explained that there was no generally accepted procedure for how to do a replication. Likewise, there was no generally accepted standard for determining whether a replication “confirms” or “disconfirms” an original study. Accordingly, the journal called for papers to identify principles for how to do a replication and how to interpret its results; and to apply those principles in crafting a replication plan for a study of the author’s choosing. The hope was that this exercise would produce some progress on “the practice of replication”. The special issue is now complete with a total of eight journal articles. This commentary places the respective articles within a common framework and identifies observations and lessons learned from the respective studies.

Suggested Citation

  • W. Robert Reed, 2018. "Takeaways from the Special Issue on The Practice of Replication," Working Papers in Economics 18/22, University of Canterbury, Department of Economics and Finance.
    • Handle: RePEc:cbt:econwp:18/22
    as

    Download full text from publisher

    File URL: https://repec.canterbury.ac.nz/cbt/econwp/1822.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hubbard, Raymond & Lindsay, R. Murray, 2013. "From significant difference to significant sameness: Proposing a paradigm shift in business research," Journal of Business Research, Elsevier, vol. 66(9), pages 1377-1388.
    2. Hannum, Randall J., 2018. "A replication plan for "Does social media reduce corruption?" (Information Economics and Policy, 2017)," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-7.
    3. Jha, Chandan Kumar & Sarangi, Sudipta, 2017. "Does social media reduce corruption?," Information Economics and Policy, Elsevier, vol. 39(C), pages 60-71.
    4. Daniels, Gerald Eric & Kakar, Venoo, 2018. "Normalized CES supply-side system approach: How to replicate Klump, McAdam, and Willman (2007)," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-11.
    5. Donald R. Haurin & Stuart S. Rosenthal, 2007. "The Influence of Household Formation on Homeownership Rates Across Time and Race," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 35(4), pages 411-450, December.
    6. Santos, Florence & Fletschner, Diana & Savath, Vivien & Peterman, Amber, 2014. "Can Government-Allocated Land Contribute to Food Security? Intrahousehold Analysis of West Bengal’s Microplot Allocation Program," World Development, Elsevier, vol. 64(C), pages 860-872.
    7. Santos, Florence & Fletschner, Diana & Savath, Vivien & Peterman, Amber, 2013. "Can government-allocated land contribute to food security? Intrahousehold analysis of West Bengal’s microplot allocation program:," IFPRI discussion papers 1310, International Food Policy Research Institute (IFPRI).
    8. Brown, Annette N. & Wood, Benjamin Douglas Kuflick, 2018. "Which tests not witch hunts: A diagnostic approach for conducting replication research," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-26.
    9. Benjamin D K Wood & Rui Müller & Annette N Brown, 2018. "Push button replication: Is impact evaluation evidence for international development verifiable?," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-15, December.
    10. Daron Acemoglu & Francisco A. Gallego & James A. Robinson, 2014. "Institutions, Human Capital, and Development ," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 875-912, August.
    11. Hannum, Randall J., 2017. "A replication plan for "Does social media reduce corruption?" (Information Economics and Policy, 2017)," Economics Discussion Papers 2017-72, Kiel Institute for the World Economy (IfW Kiel).
    12. Wood, Benjamin Douglas Kuflick & Vasquez, Maria, 2018. "Microplots and food security: Encouraging replication studies of policy relevant research," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-12.
    13. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    14. Paul Gertler & Sebastian Galiani & Mauricio Romero, 2018. "How to make replication the norm," Nature, Nature, vol. 554(7693), pages 417-419, February.
    15. Nosek, Brian A. & Ebersole, Charles R. & DeHaven, Alexander Carl & Mellor, David Thomas, 2018. "The Preregistration Revolution," OSF Preprints 2dxu5, Center for Open Science.
    16. McCullough, B. D., 2018. "Quis custodiet ipsos custodes? Despite evidence to the contrary, the American Economic Review concluded that all was well with its archive," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-13.
    17. Owen, P. Dorian, 2018. "Replication to assess statistical adequacy," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-16.
    18. Rainer Klump & Peter McAdam & Alpo Willman, 2007. "Factor Substitution and Factor-Augmenting Technical Progress in the United States: A Normalized Supply-Side System Approach," The Review of Economics and Statistics, MIT Press, vol. 89(1), pages 183-192, February.
    19. Chang, Andrew C., 2018. "A replication recipe: List your ingredients before you start cooking," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-8.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. REED: The Devil, the Deep Blue Sea, and Replication
      by replicationnetwork in The Replication Network on 2018-12-01 08:51:09

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ngoc Thi Minh Tran & Michael P. Cameron & Jacques Poot, 2021. "How Robust Is the Evidence on the Impact of Diasporas on Institutional Quality in Home Countries?," Kyklos, Wiley Blackwell, vol. 74(1), pages 126-152, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christophe Hurlin & Christophe Pérignon, 2020. "Reproducibility Certification in Economics Research," Working Papers hal-02896404, HAL.
    2. Benjamin D K Wood & Rui Müller & Annette N Brown, 2018. "Push button replication: Is impact evaluation evidence for international development verifiable?," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-15, December.
    3. Owen, P. Dorian, 2018. "Replication to assess statistical adequacy," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-16.
    4. Wood, Benjamin Douglas Kuflick & Vasquez, Maria, 2018. "Microplots and food security: Encouraging replication studies of policy relevant research," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 12, pages 1-12.
    5. My Nguyen & Kien Le, 2023. "The impacts of women's land ownership: Evidence from Vietnam," Review of Development Economics, Wiley Blackwell, vol. 27(1), pages 158-177, February.
    6. Kelly Sharp & Hisham Zerriffi & Philippe Billon, 2020. "Land scarcity, resettlement and food security: Assessing the effect of voluntary resettlement on diet quality in Malawi," Food Security: The Science, Sociology and Economics of Food Production and Access to Food, Springer;The International Society for Plant Pathology, vol. 12(1), pages 191-205, February.
    7. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    8. Srinivas Goli & Anu Rammohan & Sri Priya Reddy, 2021. "The interaction of household agricultural landholding and Caste on food security in rural Uttar Pradesh, India," Food Security: The Science, Sociology and Economics of Food Production and Access to Food, Springer;The International Society for Plant Pathology, vol. 13(1), pages 219-237, February.
    9. Meinzen-Dick, Ruth & Quisumbing, Agnes & Doss, Cheryl & Theis, Sophie, 2019. "Women's land rights as a pathway to poverty reduction: Framework and review of available evidence," Agricultural Systems, Elsevier, vol. 172(C), pages 72-82.
    10. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    11. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    12. Constanza Gonzalez Parrao & Marta Moratti & Shannon Shisler & Birte Snilstveit & John Eyers, 2021. "PROTOCOL: Aquaculture for improving productivity, income, nutrition and women's empowerment in low‐ and middle‐income countries: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(3), September.
    13. Brodeur, Abel & Esterling, Kevin & Ankel-Peters, Jörg & Bueno, Natália S. & Desposato, Scott & Dreber, Anna & Genovese, Federica & Green, Donald P. & Hepplewhite, Matthew & Hoces de la Guardia, Fernan, 2024. "Promoting Reproducibility and Replicability in Political Science," I4R Discussion Paper Series 100, The Institute for Replication (I4R).
    14. Sovacool, Benjamin K., 2018. "Bamboo Beating Bandits: Conflict, Inequality, and Vulnerability in the Political Ecology of Climate Change Adaptation in Bangladesh," World Development, Elsevier, vol. 102(C), pages 183-194.
    15. Johnson, Nancy L. & Kovarik, Chiara & Meinzen-Dick, Ruth & Njuki, Jemimah & Quisumbing, Agnes, 2016. "Gender, Assets, and Agricultural Development: Lessons from Eight Projects," World Development, Elsevier, vol. 83(C), pages 295-311.
    16. Yokying, Phanwin & Lambrecht, Isabel, 2020. "Landownership and the gender gap in agriculture: Insights from northern Ghana," Land Use Policy, Elsevier, vol. 99(C).
    17. Vu, Ha Thu & Goto, Daisaku, 2020. "Does awareness about land tenure security (LTS) increase investments in agriculture? Evidence from rural households in Vietnam," Land Use Policy, Elsevier, vol. 97(C).
    18. Abebaw, Degnet & Admassie, Assefa & Kassa, Habtemariam & Padoch, Christine, 2020. "Can rural outmigration improve household food security? Empirical evidence from Ethiopia," World Development, Elsevier, vol. 129(C).
    19. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    20. Johnson, Nancy L. & Balagamwala, Mysbah & Pinkstaff, Crossley & Theis, Sophie & Meinzen-Dick, Ruth Suseela & Quisumbing, Agnes R., 2017. "How do agricultural development projects aim to empower women?: Insights from an analysis of project strategies," IFPRI discussion papers 1609, International Food Policy Research Institute (IFPRI).

    More about this item

    Keywords

    Replication; pre-analysis plan; reproduction; repetition; extension; robustness;
    All these keywords.

    JEL classification:

    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C50 - Mathematical and Quantitative Methods - - Econometric Modeling - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cbt:econwp:18/22. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Albert Yee (email available below). General contact details of provider: https://edirc.repec.org/data/decannz.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.