IDEAS home Printed from https://ideas.repec.org/h/elg/eechap/20022_11.html
   My bibliography  Save this book chapter

Recent developments in inference: practicalities for applied economics

In: A Modern Guide to Food Economics

Author

Listed:
  • Jeffrey D. Michler
  • Anna Josephson

Abstract

The past two decades have seen phenomenal growth in applied microeconomic research. While much of the research has focused on generating unbiased estimators, there has also been substantial advances in statistical inference. We synthesize these recent advancements in calculating standard errors and test statistics for hypothesis testing in order to provide guidance for applied economists. In this chapter, we make three points. First, the applied economist needs to clearly articulate the challenges to inference that are present in the data and the source of those challenge. Second, modern computing power and statistical software means that applied economists have no excuse for not correctly calculating their standard errors. Third, it should become standard practice to rely on asymptotic refinements to the distribution of an estimator or test statistic, presenting appropriately calculated bootstrapped critical values for hypothesis testing in the main results, and not hypothesis tests that rely on first-order asymptotic theory.

Suggested Citation

  • Jeffrey D. Michler & Anna Josephson, 2022. "Recent developments in inference: practicalities for applied economics," Chapters, in: A Modern Guide to Food Economics, chapter 11, pages 235-268, Edward Elgar Publishing.
  • Handle: RePEc:elg:eechap:20022_11
    as

    Download full text from publisher

    File URL: https://www.elgaronline.com/view/edcoll/9781800372047/9781800372047.00019.xml
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Yoav Benjamini & Abba M. Krieger & Daniel Yekutieli, 2006. "Adaptive linear step-up procedures that control the false discovery rate," Biometrika, Biometrika Trust, vol. 93(3), pages 491-507, September.
    2. Anderson, Michael L, 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt15n8j26f, Department of Agricultural & Resource Economics, UC Berkeley.
    3. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.
    4. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    5. James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: evidence from the HighScope Perry Preschool Program," CeMMAP working papers CWP22/10, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. Guido W. Imbens & Michal Kolesár, 2016. "Robust Standard Errors in Small Samples: Some Practical Advice," The Review of Economics and Statistics, MIT Press, vol. 98(4), pages 701-712, October.
    7. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    8. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    9. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    10. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    11. Tim C. Hesterberg, 2015. "What Teachers Should Know About the Bootstrap: Resampling in the Undergraduate Statistics Curriculum," The American Statistician, Taylor & Francis Journals, vol. 69(4), pages 371-386, November.
    12. Alberto Abadie & Susan Athey & Guido W Imbens & Jeffrey M Wooldridge, 2023. "When Should You Adjust Standard Errors for Clustering?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(1), pages 1-35.
    13. Katherine Casey & Rachel Glennerster & Edward Miguel, 2012. "Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(4), pages 1755-1812.
    14. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2011. "Robust Inference With Multiway Clustering," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(2), pages 238-249, April.
    15. Rodrigo Pinto & Azeem Shaikh & Adam Yavitz & James Heckman, 2010. "Inference with Imperfect Randomization: The Case of the Perry Preschool Program," 2010 Meeting Papers 1336, Society for Economic Dynamics.
    16. JAMES G. MacKINNON, 2006. "Bootstrap Methods in Econometrics," The Economic Record, The Economic Society of Australia, vol. 82(s1), pages 2-18, September.
    17. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    18. Soohyung Lee & Azeem M. Shaikh, 2014. "Multiple Testing And Heterogeneous Treatment Effects: Re‐Evaluating The Effect Of Progresa On School Enrollment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(4), pages 612-626, June.
    19. J.J. Heckman & E.E. Leamer (ed.), 2001. "Handbook of Econometrics," Handbook of Econometrics, Elsevier, edition 1, volume 5, number 5.
    20. Abel Brodeur & Nikolai Cook & Anthony Heyes, 2020. "Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics," American Economic Review, American Economic Association, vol. 110(11), pages 3634-3660, November.
    21. Andrew V. Carter & Kevin T. Schnepel & Douglas G. Steigerwald, 2017. "Asymptotic Behavior of a t -Test Robust to Cluster Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 99(4), pages 698-709, July.
    22. Joel L. Horowitz, 2019. "Bootstrap Methods in Econometrics," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 193-224, August.
    23. Sarah A. Janzen & Jeffrey D. Michler, 2021. "Ulysses' pact or Ulysses' raft: Using pre‐analysis plans in experimental and nonexperimental research," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(4), pages 1286-1304, December.
    24. John List & Azeem Shaikh & Yang Xu, 2016. "Multiple Hypothesis Testing in Experimental Economics," Artefactual Field Experiments 00402, The Field Experiments Website.
    25. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    26. Jeffrey R Kling & Jeffrey B Liebman & Lawrence F Katz, 2007. "Experimental Analysis of Neighborhood Effects," Econometrica, Econometric Society, vol. 75(1), pages 83-119, January.
    27. Elizabeth Tipton & James E. Pustejovsky, 2015. "Small-Sample Adjustments for Tests of Moderators and Model Fit Using Robust Variance Estimation in Meta-Regression," Journal of Educational and Behavioral Statistics, , vol. 40(6), pages 604-634, December.
    28. James E. Pustejovsky & Elizabeth Tipton, 2018. "Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 672-683, October.
    29. Davide Viviano & Kaspar Wuthrich & Paul Niehaus, 2021. "When should you adjust inferences for multiple hypothesis testing?," Papers 2104.13367, arXiv.org, revised Dec 2023.
    30. Christopher F Baum & Mark E. Schaffer & Steven Stillman, 2007. "Enhanced routines for instrumental variables/generalized method of moments estimation and testing," Stata Journal, StataCorp LP, vol. 7(4), pages 465-506, December.
    31. Simon Heß, 2017. "Randomization inference with Stata: A guide and software," Stata Journal, StataCorp LP, vol. 17(3), pages 630-651, September.
    32. David Roodman & James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2019. "Fast and wild: Bootstrap inference in Stata using boottest," Stata Journal, StataCorp LP, vol. 19(1), pages 4-60, March.
    33. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    34. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    35. Joseph P. Romano & Azeem M. Shaikh & Michael Wolf, 2010. "Hypothesis Testing in Econometrics," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 75-104, September.
    36. Anderson, Michael L., 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1481-1495.
    37. James Heckman & Seong Hyeok Moon & Rodrigo Pinto & Peter Savelyev & Adam Yavitz, 2010. "Analyzing social experiments as implemented: A reexamination of the evidence from the HighScope Perry Preschool Program," Quantitative Economics, Econometric Society, vol. 1(1), pages 1-46, July.
    38. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    39. Marcelo Tyszler & James E. Pustejovsky & Elizabeth Tipton, 2017. "REG_SANDWICH: Stata module to compute cluster-robust (sandwich) variance estimators with small-sample corrections for linear regression," Statistical Software Components S458352, Boston College Department of Economics, revised 31 May 2017.
    40. Damian Clarke & Joseph P. Romano & Michael Wolf, 2020. "The Romano–Wolf multiple-hypothesis correction in Stata," Stata Journal, StataCorp LP, vol. 20(4), pages 812-843, December.
    41. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    42. Damian Clarke, 2016. "RWOLF: Stata module to calculate Romano-Wolf stepdown p-values for multiple hypothesis testing," Statistical Software Components S458276, Boston College Department of Economics, revised 08 Jul 2020.
    43. Peter Ganong & Simon Jäger, 2018. "A Permutation Test for the Regression Kink Design," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 494-504, April.
    44. Young, Alwyn, 2019. "Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results," LSE Research Online Documents on Economics 101401, London School of Economics and Political Science, LSE Library.
    45. MacKinnon, James G. & White, Halbert, 1985. "Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties," Journal of Econometrics, Elsevier, vol. 29(3), pages 305-325, September.
    46. Alwyn Young, 2019. "Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 557-598.
    47. McKenzie, David, 2012. "Beyond baseline and follow-up: The case for more T in experiments," Journal of Development Economics, Elsevier, vol. 99(2), pages 210-221.
    48. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    49. Andreas Steinmayr, 2020. "MHTREG: Stata module for multiple hypothesis testing controlling for FWER," Statistical Software Components S458853, Boston College Department of Economics.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marsh, W. Blake, 2023. "Supervisory stringency, payout restrictions, and bank equity prices," Journal of Banking & Finance, Elsevier, vol. 154(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    2. James G. MacKinnon & Matthew D. Webb, 2020. "When and How to Deal with Clustered Errors in Regression Models," Working Paper 1421, Economics Department, Queen's University.
    3. James G. MacKinnon, 2019. "How cluster‐robust inference is changing applied econometrics," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 52(3), pages 851-881, August.
    4. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    5. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Testing for the appropriate level of clustering in linear regression models," Journal of Econometrics, Elsevier, vol. 235(2), pages 2027-2056.
    6. Arouna, Aminou & Michler, Jeffrey D. & Lokossou, Jourdain C., 2021. "Contract farming and rural transformation: Evidence from a field experiment in Benin," Journal of Development Economics, Elsevier, vol. 151(C).
    7. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    8. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2021. "Wild Bootstrap and Asymptotic Inference With Multiway Clustering," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(2), pages 505-519, March.
    9. John A. List & Azeem M. Shaikh & Yang Xu, 2019. "Multiple hypothesis testing in experimental economics," Experimental Economics, Springer;Economic Science Association, vol. 22(4), pages 773-793, December.
    10. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Leverage, influence, and the jackknife in clustered regression models: Reliable inference using summclust," Stata Journal, StataCorp LP, vol. 23(4), pages 942-982, December.
    11. James G. MacKinnon & Matthew D. Webb, 2017. "Pitfalls When Estimating Treatment Effects Using Clustered Data," Working Paper 1387, Economics Department, Queen's University.
    12. Djogbenou, Antoine A. & MacKinnon, James G. & Nielsen, Morten Ørregaard, 2019. "Asymptotic theory and wild bootstrap inference with clustered errors," Journal of Econometrics, Elsevier, vol. 212(2), pages 393-412.
    13. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Fast and reliable jackknife and bootstrap methods for cluster‐robust inference," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 671-694, August.
    14. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    15. Brennan S Thompson & Matthew D Webb, 2019. "A simple, graphical approach to comparing multiple treatments," The Econometrics Journal, Royal Economic Society, vol. 22(2), pages 188-205.
    16. Patricia Justino & Marinella Leone & Pierfrancesco Rolla & Monique Abimpaye & Caroline Dusabe & Marie D Uwamahoro & Richard Germond, 2023. "Improving Parenting Practices for Early Child Development: Experimental Evidence from Rwanda," Journal of the European Economic Association, European Economic Association, vol. 21(4), pages 1510-1550.
    17. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    18. Guigonan S. Adjognon & Daan van Soest & Jonas Guthoff, 2021. "Reducing Hunger with Payments for Environmental Services (PES): Experimental Evidence from Burkina Faso," American Journal of Agricultural Economics, John Wiley & Sons, vol. 103(3), pages 831-857, May.
    19. Adjognon,Guigonan Serge & Nguyen Huy,Tung & Guthoff,Jonas Christoph & van Soest,Daan, 2022. "Incentivizing Social Learning for the Diffusion of Climate-Smart Agricultural Techniques," Policy Research Working Paper Series 10041, The World Bank.
    20. Ham, Andrés & Michelson, Hope C., 2018. "Does the form of delivering incentives in conditional cash transfers matter over a decade later?," Journal of Development Economics, Elsevier, vol. 134(C), pages 96-108.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:elg:eechap:20022_11. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Darrel McCalla (email available below). General contact details of provider: http://www.e-elgar.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.