IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v35y2011i3p269-303.html
   My bibliography  Save this article

Methodological Considerations in Using Complex Survey Data

Author

Listed:
  • Debbie L. Hahs-Vaughn
  • Christine M. McWayne
  • Rebecca J. Bulotsky-Shearer
  • Xiaoli Wen
  • Ann-Marie Faria

Abstract

Complex survey data are collected by means other than simple random samples. This creates two analytical issues: nonindependence and unequal selection probability. Failing to address these issues results in underestimated standard errors and biased parameter estimates. Using data from the nationally representative Head Start Family and Child Experiences Survey (FACES; 1997 and 2000 cohorts), three diverse multilevel models are presented that illustrate differences in results depending on addressing or ignoring the complex sampling issues. Limitations of using complex survey data are reported, along with recommendations for reporting complex sample results.

Suggested Citation

  • Debbie L. Hahs-Vaughn & Christine M. McWayne & Rebecca J. Bulotsky-Shearer & Xiaoli Wen & Ann-Marie Faria, 2011. "Methodological Considerations in Using Complex Survey Data," Evaluation Review, , vol. 35(3), pages 269-303, June.
    • Handle: RePEc:sae:evarev:v:35:y:2011:i:3:p:269-303
    DOI: 10.1177/0193841X11412071
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X11412071
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X11412071?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Vaughn, Gerald F., 1998. "In Review," Choices: The Magazine of Food, Farm, and Resource Issues, Agricultural and Applied Economics Association, vol. 13(2), pages 1-1.
    2. Schenker N. & Gentleman J. F., 2001. "On Judging the Significance of Differences by Examining the Overlap Between Confidence Intervals," The American Statistician, American Statistical Association, vol. 55, pages 182-186, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kull, Melissa A. & Coley, Rebekah Levine, 2015. "Early physical health conditions and school readiness skills in a prospective birth cohort of U.S. children," Social Science & Medicine, Elsevier, vol. 142(C), pages 145-153.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Domínguez-Torreiro, Marcos & Soliño, Mario, 2011. "Provided and perceived status quo in choice experiments: Implications for valuing the outputs of multifunctional rural areas," Ecological Economics, Elsevier, vol. 70(12), pages 2523-2531.
    2. Tim Goedemé & Karel Van den Bosch & Lina Salanauskaite & Gerlinde Verbist, 2013. "Testing the Statistical Significance of Microsimulation Results: Often Easier than You Think. A Technical Note," ImPRovE Working Papers 13/10, Herman Deleeck Centre for Social Policy, University of Antwerp.
    3. Yao Xiao & Chengzhen Meng & Suli Huang & Yanran Duan & Gang Liu & Shuyuan Yu & Ji Peng & Jinquan Cheng & Ping Yin, 2021. "Short-Term Effect of Temperature Change on Non-Accidental Mortality in Shenzhen, China," IJERPH, MDPI, vol. 18(16), pages 1-14, August.
    4. Thomas Yee & Alfian Hadi, 2014. "Row–column interaction models, with an R implementation," Computational Statistics, Springer, vol. 29(6), pages 1427-1445, December.
    5. Andrew Kerner & Morten Jerven & Alison Beatty, 2017. "Does it pay to be poor? Testing for systematically underreported GNI estimates," The Review of International Organizations, Springer, vol. 12(1), pages 1-38, March.
    6. McKendree, Melissa G.S. & Olynk Widmar, Nicole & Ortega, David L. & Foster, Kenneth A., 2013. "Consumer Preferences for Verified Pork-Rearing Practices in the Production of Ham Products," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 38(3), pages 1-21.
    7. Kumar, Santosh & Nahlen, Bernard, 2023. "Intergenerational persistence of health: Evidence from India," Economics Letters, Elsevier, vol. 224(C).
    8. Paul Mwebaze & Jeff Bennett & Nigel W. Beebe & Gregor J. Devine & Paul Barro, 2018. "Economic Valuation of the Threat Posed by the Establishment of the Asian Tiger Mosquito in Australia," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 71(2), pages 357-379, October.
    9. Amy E. Wagler, 2014. "Confidence Intervals for Assessing Heterogeneity in Generalized Linear Mixed Models," Journal of Educational and Behavioral Statistics, , vol. 39(3), pages 167-179, June.
    10. Laura Lindberg & Kathryn Kost & Isaac Maddow-Zimet & Sheila Desai & Mia Zolna, 2020. "Abortion Reporting in the United States: An Assessment of Three National Fertility Surveys," Demography, Springer;Population Association of America (PAA), vol. 57(3), pages 899-925, June.
    11. Tobias Keller & Martin Glaum & Andreas Bausch & Thorsten Bunz, 2023. "The “CEO in context” technique revisited: A replication and extension of Hambrick and Quigley (2014)," Strategic Management Journal, Wiley Blackwell, vol. 44(4), pages 1111-1138, April.
    12. Yuqing Feng & Jing Wei & Maogui Hu & Chengdong Xu & Tao Li & Jinfeng Wang & Wei Chen, 2022. "Lagged Effects of Exposure to Air Pollutants on the Risk of Pulmonary Tuberculosis in a Highly Polluted Region," IJERPH, MDPI, vol. 19(9), pages 1-13, May.
    13. Bir, Courtney & Wolf, Christopher A. & Widmar, Nicole Olynk, 2020. "Dog and Cat Owner Demand for Veterinary Service Payment Plans," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 46(2), August.
    14. Sabine D Klein & Loredana Torchetti & Martin Frei-Erb & Ursula Wolf, 2015. "Usage of Complementary Medicine in Switzerland: Results of the Swiss Health Survey 2012 and Development Since 2007," PLOS ONE, Public Library of Science, vol. 10(10), pages 1-10, October.
    15. Franceschini, Fiorenzo & Maisano, Domenico & Mastrogiacomo, Luca, 2014. "Scientific journal publishers and omitted citations in bibliometric databases: Any relationship?," Journal of Informetrics, Elsevier, vol. 8(3), pages 751-765.
    16. Derzon, James H. & Yu, Ping & Ellis, Bruce & Xiong, Sharon & Arroyo, Carmen & Mannix, Danyelle & Wells, Michael E. & Hill, Gary & Rollison, Julia, 2012. "A national evaluation of Safe Schools/Healthy Students: Outcomes and influences," Evaluation and Program Planning, Elsevier, vol. 35(2), pages 293-302.
    17. Carson Reeling & Leah H. Palm-Forster & Richard T. Melstrom, 2019. "Policy Instruments and Incentives for Coordinated Habitat Conservation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 73(3), pages 791-813, July.
    18. Samuel Hollander, 2002. "Economic Organization, Distribution And The Equality Issue: The Marx-Engels Perspective," Carleton Economic Papers 02-05, Carleton University, Department of Economics.
    19. Tom Tregenza & Petri T Niemelä & Rolando Rodríguez-Muñoz & Paul E Hopwood, 2022. "Environment and mate attractiveness in a wild insect [Female pied flycatchers choose territory quality and not male characteristics]," Behavioral Ecology, International Society for Behavioral Ecology, vol. 33(5), pages 999-1006.
    20. Martin Klein & Tommy Wright & Jerzy Wieczorek, 2020. "A joint confidence region for an overall ranking of populations," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(3), pages 589-606, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:35:y:2011:i:3:p:269-303. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.