IDEAS home Printed from https://ideas.repec.org/p/cbt/econwp/21-12.html
   My bibliography  Save this paper

A Replication of “The effect of the conservation reserve program on rural economies: Deriving a statistical verdict from a null finding” (American Journal of Agricultural Economics, 2019)

Author

Listed:

Abstract

This study replicates Brown, Lambert, and Wojan (2019) (BLW) and their bootstrapping procedure for calculating ex post power. At the current time there is no generally accepted way of calculating ex post power. BLW provide a novel method for doing this though they provide little justification for their method or evidence of its reliability. My replication makes three contributions. First, it confirms that the data and code provided with their paper is sufficient to reproduce their results. Second, it performs two robustness checks to determine if slight alterations to their procedure affect their results. I determine that including a constant term in their procedure does not affect the results. On the other hand, using a different bootstrapping procedure produces somewhat different results. However, without any ground truth to use as a benchmark, one cannot say which bootstrapping procedure is better. My third contribution is that I use Monte Carlo experiments to assess the performance of BLW’s method. My experimental results indicate that their method is unbiased and produces a relatively narrow range of estimates. This suggests that BLW’s method may provide a reliable method for researchers to calculate ex post power, though further investigation needs to be done.

Suggested Citation

  • Jiarui Tian, 2021. "A Replication of “The effect of the conservation reserve program on rural economies: Deriving a statistical verdict from a null finding” (American Journal of Agricultural Economics, 2019)," Working Papers in Economics 21/12, University of Canterbury, Department of Economics and Finance.
    • Handle: RePEc:cbt:econwp:21/12
    as

    Download full text from publisher

    File URL: https://repec.canterbury.ac.nz/cbt/econwp/2112.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    2. Jason P Brown & Dayton M Lambert & Timothy R Wojan, 2019. "The Effect of the Conservation Reserve Program on Rural Economies: Deriving a Statistical Verdict from a Null Finding," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 101(2), pages 528-540.
    3. Hoenig J. M. & Heisey D. M., 2001. "The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis," The American Statistician, American Statistical Association, vol. 55, pages 19-24, February.
    4. David Roodman & James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2019. "Fast and wild: Bootstrap inference in Stata using boottest," Stata Journal, StataCorp LLC, vol. 19(1), pages 4-60, March.
    5. Eric W. Djimeu & Deo-Gracias Houndolo, 2016. "Power calculation for causal inference in social science: sample size and minimum detectable effect determination," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 8(4), pages 508-527, October.
    6. Sullivan, Patrick & Hellerstein, Daniel & Hansen, LeRoy T. & Johansson, Robert C. & Koenig, Steven R. & Lubowski, Ruben N. & McBride, William D. & McGranahan, David A. & Roberts, Michael J. & Vogel, S, 2004. "The Conservation Reserve Program: Economic Implications for Rural America," Agricultural Economic Reports 33987, United States Department of Agriculture, Economic Research Service.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tian Jiarui (Alex), 2023. "A Replication of “The Effect of the Conservation Reserve Program on Rural Economies: Deriving a Statistical Verdict from a Null Finding” (American Journal of Agricultural Economics, 2019)," Economics - The Open-Access, Open-Assessment Journal, De Gruyter, vol. 17(1), pages 1-7, January.
    2. Choi, Eleanor Jawon & Choi, Jaewoo & Son, Hyelim, 2020. "The long-term effects of labor market entry in a recession: Evidence from the Asian financial crisis," Labour Economics, Elsevier, vol. 67(C).
    3. Wissmann, Daniel, 2020. "Finally a Smoking Gun," Discussion Papers in Economics 73026, University of Munich, Department of Economics.
    4. Gric, Zuzana & Ehrenbergerova, Dominika & Hodula, Martin, 2022. "The power of sentiment: Irrational beliefs of households and consumer loan dynamics," Journal of Financial Stability, Elsevier, vol. 59(C).
    5. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Fast and reliable jackknife and bootstrap methods for cluster‐robust inference," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 671-694, August.
    6. Dominika Ehrenbergerová & Martin Hodula & Zuzana Gric, 2022. "Does capital-based regulation affect bank pricing policy?," Journal of Regulatory Economics, Springer, vol. 61(2), pages 135-167, April.
    7. Seres, Gyula & Balleyer, Anna Helen & Cerutti, Nicola & Danilov, Anastasia & Friedrichsen, Jana & Liu, Yiming & Süer, Müge, 2021. "Face masks increase compliance with physical distancing recommendations during the COVID-19 pandemic," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 7(2), pages 139-158.
    8. Doremus, Jacqueline, 2019. "Unintended impacts from forest certification: Evidence from indigenous Aka households in Congo," Ecological Economics, Elsevier, vol. 166(C), pages 1-1.
    9. Meinhofer, Angélica & Witman, Allison E. & Hinde, Jesse M. & Simon, Kosali, 2021. "Marijuana liberalization policies and perinatal health," Journal of Health Economics, Elsevier, vol. 80(C).
    10. MacKinnon, James G., 2023. "Fast cluster bootstrap methods for linear regression models," Econometrics and Statistics, Elsevier, vol. 26(C), pages 52-71.
    11. Duque, Valentina & Gilraine, Michael, 2022. "Coal use, air pollution, and student performance," Journal of Public Economics, Elsevier, vol. 213(C).
    12. Bartlett, Robert P. & McCrary, Justin, 2019. "How rigged are stock markets? Evidence from microsecond timestamps," Journal of Financial Markets, Elsevier, vol. 45(C), pages 37-60.
    13. Conti, Gabriella & Poupakis, Stavros & Ekamper, Peter & Bijwaard, Govert E. & Lumey, L.H., 2024. "Severe prenatal shocks and adolescent health: Evidence from the Dutch Hunger Winter," Economics & Human Biology, Elsevier, vol. 53(C).
    14. Draca, Mirko & Garred, Jason & Stickland, Leanne & Warrinnier, Nele, 2018. "On Target? The Incidence of Sanctions Across Listed Firms in Iran," CAGE Online Working Paper Series 372, Competitive Advantage in the Global Economy (CAGE).
    15. Bargain, Olivier & Boutin, Delphine & Champeaux, Hugues, 2019. "Women's political participation and intrahousehold empowerment: Evidence from the Egyptian Arab Spring," Journal of Development Economics, Elsevier, vol. 141(C).
    16. Wang, Wenjie, 2021. "Wild Bootstrap for Instrumental Variables Regression with Weak Instruments and Few Clusters," MPRA Paper 106227, University Library of Munich, Germany.
    17. Christopher S. Carpenter & Gilbert Gonzales Jr. & Tara McKay & Dario Sansone, 2020. "Effects of the Affordable Care Act Dependent Coverage Mandate on Health Insurance Coverage for Individuals in Same-Sex Couples," NBER Working Papers 26978, National Bureau of Economic Research, Inc.
    18. García-Ramos, Aixa, 2021. "Divorce laws and intimate partner violence: Evidence from Mexico," Journal of Development Economics, Elsevier, vol. 150(C).
    19. Dupuy, Jules & Barnay, Thomas & Defebvre, Eric, 2024. "Clear as a bell? Policy stringency and elderly health during Covid-19," Social Science & Medicine, Elsevier, vol. 349(C).
    20. Gudibande, Rohan Ravindra & Jacob, Arun, 2020. "Minimum wages for domestic workers: impact evaluation of the Indian experience," World Development, Elsevier, vol. 130(C).

    More about this item

    Keywords

    Ex post power; Statistical insignificance; Monte Carlo experiments; Bootstrapping; Replication;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cbt:econwp:21/12. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Albert Yee (email available below). General contact details of provider: https://edirc.repec.org/data/decannz.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.