IDEAS home Printed from https://ideas.repec.org/a/bpj/statpp/v3y2012i1p30n2.html
   My bibliography  Save this article

Improving Statistical Inference with Clustered Data

Author

Listed:
  • Harden Jeffrey J.

    (University of North Carolina at Chapel Hill)

Abstract

Political science data often contain grouped observations, which produces unobserved "cluster effects" in statistical models. Typical solutions include (1) ignoring the impact on coefficients and only adjusting the standard errors of generalized linear models (GLM) or (2) addressing clustering in coefficient estimation while relying on a parametric assumption for the cluster effects and/or a large number of clusters for standard errors. I show that both approaches are problematic for inference. Through simulation I demonstrate that multilevel modeling (MLM) and generalized estimating equations (GEE) produce more efficient coefficients than does GLM. Next, I show that commonly-used MLM and GEE standard error methods can be biased downward, while bootstrapping by resampling clusters (BCSE) performs better, even with a misspecified error distribution and/or few clusters. I recommend the use of MLM or GEE to estimate coefficients and BCSE to estimate uncertainty, and show that this approach can produce divergent conclusions in applied research.

Suggested Citation

  • Harden Jeffrey J., 2012. "Improving Statistical Inference with Clustered Data," Statistics, Politics and Policy, De Gruyter, vol. 3(1), pages 1-30, January.
    • Handle: RePEc:bpj:statpp:v:3:y:2012:i:1:p:30:n:2
    DOI: 10.2202/2151-7509.1026
    as

    Download full text from publisher

    File URL: https://doi.org/10.2202/2151-7509.1026
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.2202/2151-7509.1026?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    2. Brambor, Thomas & Clark, William Roberts & Golder, Matt, 2006. "Understanding Interaction Models: Improving Empirical Analyses," Political Analysis, Cambridge University Press, vol. 14(1), pages 63-82, January.
    3. Bartels, Brandon L., 2009. "The Constraining Capacity of Legal Doctrine on the U.S. Supreme Court," American Political Science Review, Cambridge University Press, vol. 103(3), pages 474-495, August.
    4. Aderoju Oyefusi, 2008. "Oil and the Probability of Rebel Participation Among Youths in the Niger Delta of Nigeria," Journal of Peace Research, Peace Research Institute Oslo, vol. 45(4), pages 539-555, July.
    5. Caldeira, Gregory A & Wright, John R & Zorn, Christopher J W, 1999. "Sophisticated Voting and Gate-Keeping in the Supreme Court," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 15(3), pages 549-572, October.
    6. Zeileis, Achim, 2006. "Object-oriented Computation of Sandwich Estimators," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 16(i09).
    7. Kennedy, Peter E, 1995. "Randomization Tests in Econometrics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(1), pages 85-94, January.
    8. Green, Donald P. & Vavreck, Lynn, 2008. "Analysis of Cluster-Randomized Experiments: A Comparison of Alternative Estimation Approaches," Political Analysis, Cambridge University Press, vol. 16(2), pages 138-152, April.
    9. William D. Berry & Jacqueline H. R. DeMeritt & Justin Esarey, 2010. "Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential?," American Journal of Political Science, John Wiley & Sons, vol. 54(1), pages 248-266, January.
    10. Cameron, Charles M. & Cover, Albert D. & Segal, Jeffrey A., 1990. "Senate Voting on Supreme Court Nominees: A Neoinstitutional Model," American Political Science Review, Cambridge University Press, vol. 84(2), pages 525-534, June.
    11. Arceneaux, Kevin & Nickerson, David W., 2009. "Modeling Certainty with Clustered Data: A Comparison of Methods," Political Analysis, Cambridge University Press, vol. 17(2), pages 177-190, April.
    12. Paul Brace & Brent D. Boyea, 2008. "State Public Opinion, the Death Penalty, and the Practice of Electing Judges," American Journal of Political Science, John Wiley & Sons, vol. 52(2), pages 360-372, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jeffrey Harden & Thomas Carsey, 2012. "Balancing constituency representation and party responsiveness in the US Senate: the conditioning effect of state ideological heterogeneity," Public Choice, Springer, vol. 150(1), pages 137-154, January.
    2. Thushyanthan Baskaran & Zohal Hessami, 2017. "Political alignment and intergovernmental transfers in parliamentary systems: evidence from Germany," Public Choice, Springer, vol. 171(1), pages 75-98, April.
    3. E. Keith Smith & Adam Mayer, 2019. "Anomalous Anglophones? Contours of free market ideology, political polarization, and climate change attitudes in English-speaking countries, Western European and post-Communist states," Climatic Change, Springer, vol. 152(1), pages 17-34, January.
    4. von Hippel, Paul T. & Bellows, Laura & Osborne, Cynthia & Lincove, Jane Arnold & Mills, Nick, 2016. "Teacher quality differences between teacher preparation programs: How big? How reliable? Which programs are different?," Economics of Education Review, Elsevier, vol. 53(C), pages 31-45.
    5. Adam William Chalmers & Lisa Maria Dellmuth, 2015. "Fiscal redistribution and public support for European integration," European Union Politics, , vol. 16(3), pages 386-407, September.
    6. Sviták, Jan & Tichem, Jan & Haasbeek, Stefan, 2021. "Price effects of search advertising restrictions," International Journal of Industrial Organization, Elsevier, vol. 77(C).
    7. Chwieroth, Jeffrey & Walter, Andrew, 2015. "Great expectations, veto players, and the changing politics of banking crises," LSE Research Online Documents on Economics 60953, London School of Economics and Political Science, LSE Library.
    8. Orkhan Sariyev & Tim K. Loos & Ling Yee Khor, 2021. "Intra-household decision-making, production diversity, and dietary quality: a panel data analysis of Ethiopian rural households," Food Security: The Science, Sociology and Economics of Food Production and Access to Food, Springer;The International Society for Plant Pathology, vol. 13(1), pages 181-197, February.
    9. Lee Epstein & Eric A. Posner, 2016. "Supreme Court Justices' Loyalty to the President," The Journal of Legal Studies, University of Chicago Press, vol. 45(2), pages 401-436.
    10. Heiko T. Burret & Lars P. Feld, 2016. "Effects of Fiscal Rules - 85 Years' Experience in Switzerland," CESifo Working Paper Series 6063, CESifo.
    11. Khalid Sekkat, 2016. "Export Orientation and Exchange Rate Changes: Do Firms React Differently and Why?," Working Papers 989, Economic Research Forum, revised Apr 2016.
    12. Annette Hofmann & Julia K. Neumann & Peter Zweifel, 2019. "Risky health decisions under regulatory constraints: Abortion tourism in Switzerland," Journal of Risk and Uncertainty, Springer, vol. 59(3), pages 203-237, December.
    13. Patrick Bayer & Johannes Urpelainen, 2013. "External sources of clean technology: Evidence from the Clean Development Mechanism," The Review of International Organizations, Springer, vol. 8(1), pages 81-109, March.
    14. Schoenherr, Jessica A. & Black, Ryan C., 2019. "Friends with benefits: Case significance, amicus curiae, and agenda setting on the U.S. Supreme Court," International Review of Law and Economics, Elsevier, vol. 58(C), pages 43-53.
    15. Steijvers, Tensie & Niskanen, Mervi, 2014. "Tax aggressiveness in private family firms: An agency perspective," Journal of Family Business Strategy, Elsevier, vol. 5(4), pages 347-357.
    16. Burret, Heiko T. & Feld, Lars P., 2018. "(Un-)intended effects of fiscal rules," European Journal of Political Economy, Elsevier, vol. 52(C), pages 166-191.
    17. Kenneth Shepsle & Barry Weingast, 2012. "Why so much stability? Majority voting, legislative institutions, and Gordon Tullock," Public Choice, Springer, vol. 152(1), pages 83-95, July.
    18. Carbó-Valverde, Santiago & Cuadros-Solas, Pedro J. & Rodríguez-Fernández, Francisco, 2017. "Do banks and industrial companies have equal access to reputable underwriters in debt markets?," Journal of Corporate Finance, Elsevier, vol. 45(C), pages 176-202.
    19. Charles R. Shipan, 2008. "Partisanship, Ideology, and Senate Voting on Supreme Court Nominees," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 5(1), pages 55-76, March.
    20. Holm-Hadulla, Fédéric, 2020. "Fiscal equalization and the tax structure," Regional Science and Urban Economics, Elsevier, vol. 81(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:statpp:v:3:y:2012:i:1:p:30:n:2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.