IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v47y2014icp26-34.html
   My bibliography  Save this article

Sensitivity analyses for clustered data: An illustration from a large-scale clustered randomized controlled trial in education

Author

Listed:
  • Abe, Yasuyo
  • Gee, Kevin A.

Abstract

In this paper, we demonstrate the importance of conducting well-thought-out sensitivity analyses for handling clustered data (data in which individuals are grouped into higher order units, such as students in schools) that arise from cluster randomized controlled trials (RCTs). This is particularly relevant given the rise in rigorous impact evaluations that use cluster randomized designs across various fields including education, public health and social welfare. Using data from a recently completed cluster RCT of a school-based teacher professional development program, we demonstrate our use of four commonly applied methods for analyzing clustered data. These methods include: (1) hierarchical linear modeling (HLM); (2) feasible generalized least squares (FGLS); (3) generalized estimating equations (GEE); and (4) ordinary least squares (OLS) regression with cluster-robust (Huber–White) standard errors. We compare our findings across each method, showing how inconsistent results – in terms of both effect sizes and statistical significance – emerged across each method and our analytic approach to resolving such inconsistencies.

Suggested Citation

  • Abe, Yasuyo & Gee, Kevin A., 2014. "Sensitivity analyses for clustered data: An illustration from a large-scale clustered randomized controlled trial in education," Evaluation and Program Planning, Elsevier, vol. 47(C), pages 26-34.
    • Handle: RePEc:eee:epplan:v:47:y:2014:i:c:p:26-34
    DOI: 10.1016/j.evalprogplan.2014.07.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S014971891400072X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2014.07.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 318, University of California, Davis, Department of Economics.
    2. repec:mpr:mprres:6371 is not listed on IDEAS
    3. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
    4. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    5. Paul Gertler, 2004. "Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA's Control Randomized Experiment," American Economic Review, American Economic Association, vol. 94(2), pages 336-341, May.
    6. Peter Z. Schochet, "undated". "Statistical Power for Random Assignment Evaluations of Education Programs," Mathematica Policy Research Reports 6749d31ad72d4acf988f7dce5, Mathematica Policy Research.
    7. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    8. repec:mpr:mprres:5863 is not listed on IDEAS
    9. Arceneaux, Kevin & Nickerson, David W., 2009. "Modeling Certainty with Clustered Data: A Comparison of Methods," Political Analysis, Cambridge University Press, vol. 17(2), pages 177-190, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gee, Kevin A., 2015. "Achieving gender equality in learning outcomes: Evidence from a non-formal education program in Bangladesh," International Journal of Educational Development, Elsevier, vol. 40(C), pages 207-216.
    2. Katharine S. Gries & Dean A. Regier & Scott D. Ramsey & Donald L. Patrick, 2017. "Utility Estimates of Disease-Specific Health States in Prostate Cancer from Three Different Perspectives," Applied Health Economics and Health Policy, Springer, vol. 15(3), pages 375-384, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    2. Richard H. Spady & Sami Stouli, 2018. "Simultaneous Mean-Variance Regression," Bristol Economics Discussion Papers 18/697, School of Economics, University of Bristol, UK.
    3. Ulrich Gunter & Irem Önder, 2018. "Determinants of Airbnb demand in Vienna and their implications for the traditional accommodation industry," Tourism Economics, , vol. 24(3), pages 270-293, May.
    4. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2017. "Sampling-based vs. Design-based Uncertainty in Regression Analysis," Papers 1706.01778, arXiv.org, revised Jun 2019.
    5. Entorf, Horst & Sattarova, Liliya, 2016. "The Analysis of Prison-Prisoner Data Using Cluster-Sample Econometrics: Prison Conditions and Prisoners' Assessments of the Future," IZA Discussion Papers 10209, Institute of Labor Economics (IZA).
    6. Haenssgen, Marco J. & Savage, Jessica & Yeboah, Godwin & Charoenboon, Nutcha & Srenh, Sorn, 2021. "In a network of lines that intersect: The socio-economic development impact of marine resource management and conservation in Southeast Asia," World Development, Elsevier, vol. 146(C).
    7. Luiz Paulo Fávero & Joseph F. Hair & Rafael de Freitas Souza & Matheus Albergaria & Talles V. Brugni, 2021. "Zero-Inflated Generalized Linear Mixed Models: A Better Way to Understand Data Relationships," Mathematics, MDPI, vol. 9(10), pages 1-28, May.
    8. Gregory, Allan W. & McCurdy, Thomas H., 1986. "The unbiasedness hypothesis in the forward foreign exchange market: A specification analysis with application to France, Italy, Japan, the United Kingdom and West Germany," European Economic Review, Elsevier, vol. 30(2), pages 365-381, April.
    9. Tesfaye, Wondimagegn & Tirivayi, Nyasha, 2020. "Crop diversity, household welfare and consumption smoothing under risk: Evidence from rural Uganda," World Development, Elsevier, vol. 125(C).
    10. Kaiser, Lutz C., 2014. "The Gender-Career Estimation Gap," IZA Discussion Papers 8185, Institute of Labor Economics (IZA).
    11. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    12. Reischmann, Markus, 2016. "Creative accounting and electoral motives: Evidence from OECD countries," Journal of Comparative Economics, Elsevier, vol. 44(2), pages 243-257.
    13. Subal C. Kumbhakar & Christopher F. Parmeter & Valentin Zelenyuk, 2022. "Stochastic Frontier Analysis: Foundations and Advances I," Springer Books, in: Subhash C. Ray & Robert G. Chambers & Subal C. Kumbhakar (ed.), Handbook of Production Economics, chapter 8, pages 331-370, Springer.
    14. Towfiqul Islam Khan & Mashfique Ibne Akbar, 2015. "Illicit Financial Flow in view of Financing the Post-2015 Development Agenda," Southern Voice Occasional Paper 25, Southern Voice.
    15. Potrafke, Niklas, 2019. "Electoral cycles in perceived corruption: International empirical evidence," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 215-224.
    16. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
    17. Davidson, Russell & MacKinnon, James G., 1989. "Testing for Consistency using Artificial Regressions," Econometric Theory, Cambridge University Press, vol. 5(3), pages 363-384, December.
    18. Chung‐Hua Shen & Hsing‐Hua Hsu, 2022. "The determinants of Asian banking crises—Application of the panel threshold logit model," International Review of Finance, International Review of Finance Ltd., vol. 22(1), pages 248-277, March.
    19. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    20. Ali Uyar & Simone Pizzi & Fabio Caputo & Cemil Kuzey & Abdullah S. Karaman, 2022. "Do shareholders reward or punish risky firms due to CSR reporting and assurance?," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 43(5), pages 1596-1620, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:47:y:2014:i:c:p:26-34. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.