IDEAS home Printed from https://ideas.repec.org/p/cbt/econwp/21-12.html
   My bibliography  Save this paper

A Replication of “The effect of the conservation reserve program on rural economies: Deriving a statistical verdict from a null finding” (American Journal of Agricultural Economics, 2019)

Author

Listed:

Abstract

This study replicates Brown, Lambert, and Wojan (2019) (BLW) and their bootstrapping procedure for calculating ex post power. At the current time there is no generally accepted way of calculating ex post power. BLW provide a novel method for doing this though they provide little justification for their method or evidence of its reliability. My replication makes three contributions. First, it confirms that the data and code provided with their paper is sufficient to reproduce their results. Second, it performs two robustness checks to determine if slight alterations to their procedure affect their results. I determine that including a constant term in their procedure does not affect the results. On the other hand, using a different bootstrapping procedure produces somewhat different results. However, without any ground truth to use as a benchmark, one cannot say which bootstrapping procedure is better. My third contribution is that I use Monte Carlo experiments to assess the performance of BLW’s method. My experimental results indicate that their method is unbiased and produces a relatively narrow range of estimates. This suggests that BLW’s method may provide a reliable method for researchers to calculate ex post power, though further investigation needs to be done.

Suggested Citation

  • Jiarui Tian, 2021. "A Replication of “The effect of the conservation reserve program on rural economies: Deriving a statistical verdict from a null finding” (American Journal of Agricultural Economics, 2019)," Working Papers in Economics 21/12, University of Canterbury, Department of Economics and Finance.
    • Handle: RePEc:cbt:econwp:21/12
    as

    Download full text from publisher

    File URL: https://repec.canterbury.ac.nz/cbt/econwp/2112.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jason P Brown & Dayton M Lambert & Timothy R Wojan, 2019. "The Effect of the Conservation Reserve Program on Rural Economies: Deriving a Statistical Verdict from a Null Finding," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 101(2), pages 528-540.
    2. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    3. Eric W. Djimeu & Deo-Gracias Houndolo, 2016. "Power calculation for causal inference in social science: sample size and minimum detectable effect determination," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 8(4), pages 508-527, October.
    4. David Roodman & James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2019. "Fast and wild: Bootstrap inference in Stata using boottest," Stata Journal, StataCorp LP, vol. 19(1), pages 4-60, March.
    5. Sullivan, Patrick & Hellerstein, Daniel & Hansen, LeRoy T. & Johansson, Robert C. & Koenig, Steven R. & Lubowski, Ruben N. & McBride, William D. & McGranahan, David A. & Roberts, Michael J. & Vogel, S, 2004. "The Conservation Reserve Program: Economic Implications for Rural America," Agricultural Economic Reports 33987, United States Department of Agriculture, Economic Research Service.
    6. Hoenig J. M. & Heisey D. M., 2001. "The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis," The American Statistician, American Statistical Association, vol. 55, pages 19-24, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tian Jiarui (Alex), 2023. "A Replication of “The Effect of the Conservation Reserve Program on Rural Economies: Deriving a Statistical Verdict from a Null Finding” (American Journal of Agricultural Economics, 2019)," Economics - The Open-Access, Open-Assessment Journal, De Gruyter, vol. 17(1), pages 1-7, January.
    2. Wissmann, Daniel, 2020. "Finally a Smoking Gun," Discussion Papers in Economics 73026, University of Munich, Department of Economics.
    3. Carpenter, Christopher S. & Gonzales, Gilbert & McKay, Tara & Sansone, Dario, 2020. "Effects of the Affordable Care Act Dependent Coverage Mandate on Health Insurance Coverage for Individuals in Same-Sex Couples," IZA Discussion Papers 13119, Institute of Labor Economics (IZA).
    4. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Fast and reliable jackknife and bootstrap methods for cluster‐robust inference," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 671-694, August.
    5. Gric, Zuzana & Ehrenbergerova, Dominika & Hodula, Martin, 2022. "The power of sentiment: Irrational beliefs of households and consumer loan dynamics," Journal of Financial Stability, Elsevier, vol. 59(C).
    6. Winters, John V. & Cai, Zhengyu & Maguire, Karen & Sengupta, Shruti, 2019. "Do Workers Benefit from Resource Booms in Their Home State? Evidence from the Fracking Era," GLO Discussion Paper Series 400, Global Labor Organization (GLO).
    7. Solórzano Diego & Dixon Huw, 2020. "The Relationship Between Nominal Wage and Price Flexibility: New Evidence," Working Papers 2020-20, Banco de México.
    8. Maria Rosaria Alfano & Anna Laura Baraldi & Claudia Cantabene, 2023. "Eppur si muove: an evaluation of museum policy reform in Italy," Journal of Cultural Economics, Springer;The Association for Cultural Economics International, vol. 47(1), pages 97-131, March.
    9. Dominika Ehrenbergerová & Martin Hodula & Zuzana Gric, 2022. "Does capital-based regulation affect bank pricing policy?," Journal of Regulatory Economics, Springer, vol. 61(2), pages 135-167, April.
    10. Bietenbeck, Jan & Leibing, Andreas & Marcus, Jan & Weinhardt, Felix, 2023. "Tuition fees and educational attainment," European Economic Review, Elsevier, vol. 154(C).
    11. Wenjie Wang & Yichong Zhang, 2021. "Wild Bootstrap for Instrumental Variables Regressions with Weak and Few Clusters," Papers 2108.13707, arXiv.org, revised Jan 2024.
    12. Damian Clarke & Kathya Tapia-Schythe, 2021. "Implementing the panel event study," Stata Journal, StataCorp LP, vol. 21(4), pages 853-884, December.
    13. James G. MacKinnon, 2019. "How cluster‐robust inference is changing applied econometrics," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 52(3), pages 851-881, August.
    14. Seres, Gyula & Balleyer, Anna Helen & Cerutti, Nicola & Danilov, Anastasia & Friedrichsen, Jana & Liu, Yiming & Süer, Müge, 2021. "Face masks increase compliance with physical distancing recommendations during the COVID-19 pandemic," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 7(2), pages 139-158.
    15. Arrieta Vidal, Johar & Florián Hoyle, David & López Vargas, Kristian & Morales Vásquez, Valeria, 2022. "Policies for transactional de-dollarization: A laboratory study," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 31-54.
    16. Doremus, Jacqueline, 2019. "Unintended impacts from forest certification: Evidence from indigenous Aka households in Congo," Ecological Economics, Elsevier, vol. 166(C), pages 1-1.
    17. Bahar, Dany & Ibáñez, Ana María & Rozo, Sandra V., 2021. "Give me your tired and your poor: Impact of a large-scale amnesty program for undocumented refugees," Journal of Development Economics, Elsevier, vol. 151(C).
    18. Meinhofer, Angélica & Witman, Allison E. & Hinde, Jesse M. & Simon, Kosali, 2021. "Marijuana liberalization policies and perinatal health," Journal of Health Economics, Elsevier, vol. 80(C).
    19. MacKinnon, James G., 2023. "Fast cluster bootstrap methods for linear regression models," Econometrics and Statistics, Elsevier, vol. 26(C), pages 52-71.
    20. Duque, Valentina & Gilraine, Michael, 2022. "Coal use, air pollution, and student performance," Journal of Public Economics, Elsevier, vol. 213(C).

    More about this item

    Keywords

    Ex post power; Statistical insignificance; Monte Carlo experiments; Bootstrapping; Replication;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cbt:econwp:21/12. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Albert Yee (email available below). General contact details of provider: https://edirc.repec.org/data/decannz.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.