IDEAS home Printed from https://ideas.repec.org/a/bpj/statpp/v3y2012i1p25n1.html
   My bibliography  Save this article

Problems with Tests of the Missingness Mechanism in Quantitative Policy Studies

Author

Listed:
  • Rhoads Christopher H.

    (University of Connecticut)

Abstract

Policy analysts involved in quantitative research have many options for handling missing data. The method chosen will often greatly influence the substantive policy conclusions that will be drawn from the data. The most frequent methods for handling missing data assume that the data are missing at random (MAR). The current paper notes that an omnibus, nonparametric test of the MAR assumption is impossible using the observed data alone. Nonetheless various purported tests of the missingness mechanism (including tests of MAR) appear in the literature. The current paper clarifies that all of these tests rely on some assumption that cannot be tested from the data. The paper notes that tests of the missingness mechanism are frequently misinterpreted and it clarifies the appropriate interpretation of such tests. Policy analysts are encouraged not to develop the false impression that modern procedures for handling missing data in conjunction with tests of the missingness mechanism provide protection against the ill effects of missing data. Any justification for a particular approach to handling missing data must be come from substantive knowledge of the missingness process, not from the data.

Suggested Citation

  • Rhoads Christopher H., 2012. "Problems with Tests of the Missingness Mechanism in Quantitative Policy Studies," Statistics, Politics and Policy, De Gruyter, vol. 3(1), pages 1-25, March.
    • Handle: RePEc:bpj:statpp:v:3:y:2012:i:1:p:25:n:1
    DOI: 10.1515/2151-7509.1012
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/2151-7509.1012
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/2151-7509.1012?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Geert Molenberghs & Caroline Beunckens & Cristina Sotto & Michael G. Kenward, 2008. "Every missingness not at random model has a missingness at random counterpart with equal fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(2), pages 371-388, April.
    2. John V. Pepper, 2000. "The Intergenerational Transmission Of Welfare Receipt: A Nonparametric Bounds Analysis," The Review of Economics and Statistics, MIT Press, vol. 82(3), pages 472-488, August.
    3. Charles F. Manski, 1989. "Anatomy of the Selection Problem," Journal of Human Resources, University of Wisconsin Press, vol. 24(3), pages 343-360.
    4. Libertad González, 2005. "Nonparametric bounds on the returns to language skills," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(6), pages 771-795.
    5. James Heckman, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    6. William N. Evans & Matthew C. Farrelly, 1998. "The Compensating Behavior of Smokers: Taxes, Tar, and Nicotine," RAND Journal of Economics, The RAND Corporation, vol. 29(3), pages 578-595, Autumn.
    7. Hedeker, Donald, 1999. "MIXNO: a computer program for mixed-effects nominal logistic regression," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 4(i05).
    8. Frieder R. Lang & Paul B. Baltes & Gert G. Wagner, 2007. "Desired Lifetime and End-of-Life Desires Across Adulthood From 20 to 90: A Dual-Source Information Model," The Journals of Gerontology: Series B, The Gerontological Society of America, vol. 62(5), pages 268-276.
    9. Buckler, Kevin & Unnever, James D., 2008. "Racial and ethnic perceptions of injustice: Testing the core hypotheses of comparative conflict theory," Journal of Criminal Justice, Elsevier, vol. 36(3), pages 270-278, July.
    10. Daniel O. Scharfstein & Charles F. Manski & James C. Anthony, 2004. "On the Construction of Bounds in Prospective Studies with Missing Ordinal Outcomes: Application to the Good Behavior Game Trial," Biometrics, The International Biometric Society, vol. 60(1), pages 154-164, March.
    11. Christine E. Grella & Christy K. Scott & Mark A. Foss & Michael L. Dennis, 2008. "Gender Similarities and Differences in the Treatment, Relapse, and Recovery Cycle," Evaluation Review, , vol. 32(1), pages 113-137, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. A.Y. Kombo & H. Mwambi & G. Molenberghs, 2017. "Multiple imputation for ordinal longitudinal data with monotone missing data patterns," Journal of Applied Statistics, Taylor & Francis Journals, vol. 44(2), pages 270-287, January.
    2. Richard M. Golden & Steven S. Henley & Halbert White & T. Michael Kashner, 2019. "Consequences of Model Misspecification for Maximum Likelihood Estimation with Missing Data," Econometrics, MDPI, vol. 7(3), pages 1-27, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Deniz Dutz & Ingrid Huitfeldt & Santiago Lacouture & Magne Mogstad & Alexander Torgovitsky & Winnie van Dijk, 2021. "Selection in Surveys: Using Randomized Incentives to Detect and Account for Nonresponse Bias," NBER Working Papers 29549, National Bureau of Economic Research, Inc.
    2. Mogstad, Magne & Torgovitsky, Alexander, 2024. "Instrumental variables with unobserved heterogeneity in treatment effects," Handbook of Labor Economics,, Elsevier.
    3. Breunig, Christoph & Mammen, Enno & Simoni, Anna, 2018. "Nonparametric estimation in case of endogenous selection," Journal of Econometrics, Elsevier, vol. 202(2), pages 268-285.
    4. Vikesh Amin & Jere R. Behrman & Jason M. Fletcher & Carlos A. Flores & Alfonso Flores-Lagunes & Hans-Peter Kohler, 2022. "Does Schooling Improve Cognitive Abilities at Older Ages: Causal Evidence from Nonparametric Bounds," PIER Working Paper Archive 22-016, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    5. Claudia Olivetti & Barbara Petrongolo, 2008. "Unequal Pay or Unequal Employment? A Cross-Country Analysis of Gender Gaps," Journal of Labor Economics, University of Chicago Press, vol. 26(4), pages 621-654, October.
    6. Ho, Kate & Rosen, Adam M., 2015. "Partial Identification in Applied Research: Benefits and Challenges," CEPR Discussion Papers 10883, C.E.P.R. Discussion Papers.
    7. Harmon, Colm & Walker, Ian, 1995. "Estimates of the Economic Return to Schooling for the United Kingdom," American Economic Review, American Economic Association, vol. 85(5), pages 1278-1286, December.
    8. Rosalia Vazquez-Alvarez, 2003. "Anchoring Bias and Covariate Nonresponse," University of St. Gallen Department of Economics working paper series 2003 2003-19, Department of Economics, University of St. Gallen.
    9. Victor Chernozhukov & Sokbae Lee & Adam M. Rosen, 2013. "Intersection Bounds: Estimation and Inference," Econometrica, Econometric Society, vol. 81(2), pages 667-737, March.
    10. Marjan Petreski & Nikica Blazevski & Blagica Petreski, 2014. "Gender Wage Gap when Women are Highly Inactive: Evidence from Repeated Imputations with Macedonian Data," Journal of Labor Research, Springer, vol. 35(4), pages 393-411, December.
    11. Shoshana Neuman & Ronald Oaxaca, 2004. "Wage Decompositions with Selectivity-Corrected Wage Equations: A Methodological Note," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 2(1), pages 3-10, April.
    12. Bo E. Honoré & Luojia Hu, 2020. "Selection Without Exclusion," Econometrica, Econometric Society, vol. 88(3), pages 1007-1029, May.
    13. Kaya, Ezgi, 2014. "Gender Wage Gap Trends in Europe: The Role of Occupational Allocation and Skill Prices," Cardiff Economics Working Papers E2014/23, Cardiff University, Cardiff Business School, Economics Section.
    14. Victor Chernozhukov & Ivan Fernandez-Val & Siyi Luo, 2023. "Distribution regression with sample selection and UK wage decomposition," CeMMAP working papers 09/23, Institute for Fiscal Studies.
    15. Adrián Rubli, 2012. "La importancia de corregir por el sesgo de selección en el análisis de las brechas salariales por género: un estudio para Argentina, Brasil y México," Ensayos Revista de Economia, Universidad Autonoma de Nuevo Leon, Facultad de Economia, vol. 0(2), pages 1-36, November.
    16. Verbeek, Marno & Nijman, Theo, 1992. "Testing for Selectivity Bias in Panel Data Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 33(3), pages 681-703, August.
    17. Antonio Di Paolo & Josep Lluís Raymond, 2012. "Language Knowledge and Earnings in Catalonia," Journal of Applied Economics, Taylor & Francis Journals, vol. 15(1), pages 89-118, May.
    18. Keith A. Bender & Robert Elliott, 1999. "Relative Earnings in the UK Public Sector: The Impact of Pay Reform on Pay Structure," Palgrave Macmillan Books, in: Robert Elliott & Claudio Lucifora & Dominique Meurs (ed.), Public Sector Pay Determination in the European Union, chapter 8, pages 285-339, Palgrave Macmillan.
    19. D. Nitsch & B. L. DeStavola & S. M. B. Morton & D. A. Leon, 2006. "Linkage bias in estimating the association between childhood exposures and propensity to become a mother: an example of simple sensitivity analyses," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(3), pages 493-505, July.
    20. Monique De Haan & Ragnhild C Schreiner, 2025. "The Intergenerational Transmission of Welfare Dependency," The Economic Journal, Royal Economic Society, vol. 135(669), pages 1609-1640.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:statpp:v:3:y:2012:i:1:p:25:n:1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyterbrill.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.