IDEAS home Printed from https://ideas.repec.org/a/bpj/statpp/v3y2012i1p25n1.html
   My bibliography  Save this article

Problems with Tests of the Missingness Mechanism in Quantitative Policy Studies

Author

Listed:
  • Rhoads Christopher H.

    (University of Connecticut)

Abstract

Policy analysts involved in quantitative research have many options for handling missing data. The method chosen will often greatly influence the substantive policy conclusions that will be drawn from the data. The most frequent methods for handling missing data assume that the data are missing at random (MAR). The current paper notes that an omnibus, nonparametric test of the MAR assumption is impossible using the observed data alone. Nonetheless various purported tests of the missingness mechanism (including tests of MAR) appear in the literature. The current paper clarifies that all of these tests rely on some assumption that cannot be tested from the data. The paper notes that tests of the missingness mechanism are frequently misinterpreted and it clarifies the appropriate interpretation of such tests. Policy analysts are encouraged not to develop the false impression that modern procedures for handling missing data in conjunction with tests of the missingness mechanism provide protection against the ill effects of missing data. Any justification for a particular approach to handling missing data must be come from substantive knowledge of the missingness process, not from the data.

Suggested Citation

  • Rhoads Christopher H., 2012. "Problems with Tests of the Missingness Mechanism in Quantitative Policy Studies," Statistics, Politics and Policy, De Gruyter, vol. 3(1), pages 1-25, March.
    • Handle: RePEc:bpj:statpp:v:3:y:2012:i:1:p:25:n:1
    DOI: 10.1515/2151-7509.1012
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/2151-7509.1012
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/2151-7509.1012?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John V. Pepper, 2000. "The Intergenerational Transmission Of Welfare Receipt: A Nonparametric Bounds Analysis," The Review of Economics and Statistics, MIT Press, vol. 82(3), pages 472-488, August.
    2. Charles F. Manski, 1989. "Anatomy of the Selection Problem," Journal of Human Resources, University of Wisconsin Press, vol. 24(3), pages 343-360.
    3. William N. Evans & Matthew C. Farrelly, 1998. "The Compensating Behavior of Smokers: Taxes, Tar, and Nicotine," RAND Journal of Economics, The RAND Corporation, vol. 29(3), pages 578-595, Autumn.
    4. Libertad González, 2005. "Nonparametric bounds on the returns to language skills," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(6), pages 771-795.
    5. Geert Molenberghs & Caroline Beunckens & Cristina Sotto & Michael G. Kenward, 2008. "Every missingness not at random model has a missingness at random counterpart with equal fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(2), pages 371-388, April.
    6. Buckler, Kevin & Unnever, James D., 2008. "Racial and ethnic perceptions of injustice: Testing the core hypotheses of comparative conflict theory," Journal of Criminal Justice, Elsevier, vol. 36(3), pages 270-278, July.
    7. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    8. Daniel O. Scharfstein & Charles F. Manski & James C. Anthony, 2004. "On the Construction of Bounds in Prospective Studies with Missing Ordinal Outcomes: Application to the Good Behavior Game Trial," Biometrics, The International Biometric Society, vol. 60(1), pages 154-164, March.
    9. Hedeker, Donald, 1999. "MIXNO: a computer program for mixed-effects nominal logistic regression," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 4(i05).
    10. Christine E. Grella & Christy K. Scott & Mark A. Foss & Michael L. Dennis, 2008. "Gender Similarities and Differences in the Treatment, Relapse, and Recovery Cycle," Evaluation Review, , vol. 32(1), pages 113-137, February.
    11. Frieder R. Lang & Paul B. Baltes & Gert G. Wagner, 2007. "Desired Lifetime and End-of-Life Desires Across Adulthood From 20 to 90: A Dual-Source Information Model," The Journals of Gerontology: Series B, The Gerontological Society of America, vol. 62(5), pages 268-276.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Richard M. Golden & Steven S. Henley & Halbert White & T. Michael Kashner, 2019. "Consequences of Model Misspecification for Maximum Likelihood Estimation with Missing Data," Econometrics, MDPI, vol. 7(3), pages 1-27, September.
    2. A.Y. Kombo & H. Mwambi & G. Molenberghs, 2017. "Multiple imputation for ordinal longitudinal data with monotone missing data patterns," Journal of Applied Statistics, Taylor & Francis Journals, vol. 44(2), pages 270-287, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David S. Lee, 2002. "Trimming for Bounds on Treatment Effects with Missing Outcomes," NBER Technical Working Papers 0277, National Bureau of Economic Research, Inc.
    2. Victor Chernozhukov & Sokbae Lee & Adam M. Rosen, 2013. "Intersection Bounds: Estimation and Inference," Econometrica, Econometric Society, vol. 81(2), pages 667-737, March.
    3. Victor Chernozhukov & Iv'an Fern'andez-Val & Siyi Luo, 2018. "Distribution Regression with Sample Selection, with an Application to Wage Decompositions in the UK," Papers 1811.11603, arXiv.org, revised Dec 2023.
    4. Martin Huber & Giovanni Mellace, 2015. "Sharp Bounds on Causal Effects under Sample Selection," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(1), pages 129-151, February.
    5. Antonio Di Paolo & Josep Lluís Raymond, 2012. "Language Knowledge and Earnings in Catalonia," Journal of Applied Economics, Taylor & Francis Journals, vol. 15(1), pages 89-118, May.
    6. Vikesh Amin & Jere R. Behrman & Jason M. Fletcher & Carlos A. Flores & Alfonso Flores-Lagunes & Hans-Peter Kohler, 2022. "Does Schooling Improve Cognitive Abilities at Older Ages: Causal Evidence from Nonparametric Bounds," PIER Working Paper Archive 22-016, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    7. van der Klaauw, Bas & Koning, Ruud H, 2003. "Testing the Normality Assumption in the Sample Selection Model with an Application to Travel Demand," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(1), pages 31-42, January.
    8. Humphreys, Keith & Phibbs, Ciaran S. & Moos, Rudolf H., 1996. "Addressing self-selection effects in evaluations of mutual help groups and professional mental health services: An introduction to two-stage sample selection models," Evaluation and Program Planning, Elsevier, vol. 19(4), pages 301-308, November.
    9. Vazquez-Alvarez, R. & Melenberg, B. & van Soest, A.H.O., 1999. "Nonparametric Modeling of the Anchoring Effect in an Unfolding Bracket Design," Discussion Paper 1999-115, Tilburg University, Center for Economic Research.
    10. Breunig, Christoph & Mammen, Enno & Simoni, Anna, 2018. "Nonparametric estimation in case of endogenous selection," Journal of Econometrics, Elsevier, vol. 202(2), pages 268-285.
    11. Kajal Lahiri & Xian Li, 2020. "Smoking Behavior of Older Adults: A Panel Data Analysis Using HRS," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 18(3), pages 495-523, September.
    12. Guilhem Bascle, 2008. "Controlling for endogeneity with instrumental variables in strategic management research," Post-Print hal-00576795, HAL.
    13. Kamila Cygan‐Rehm & Daniel Kuehnle & Michael Oberfichtner, 2017. "Bounding the causal effect of unemployment on mental health: Nonparametric evidence from four countries," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1844-1861, December.
    14. Brick J. Michael, 2013. "Unit Nonresponse and Weighting Adjustments: A Critical Review," Journal of Official Statistics, Sciendo, vol. 29(3), pages 329-353, June.
    15. Geraint Johnes, 1999. "Schooling, fertility and the labour market experience of married women," Applied Economics, Taylor & Francis Journals, vol. 31(5), pages 585-592.
    16. Christelis, Dimitris & Messina, Julián, 2019. "Partial Identification of Population Average and Quantile Treatment Effects in Observational Data under Sample Selection," IDB Publications (Working Papers) 9520, Inter-American Development Bank.
    17. Ibáñez, Ana María & Muñoz, Juan Carlos & Verwimp, Philip, 2013. "Abandoning Coffee under the Threat of Violence and the Presence of Illicit Crops. Evidence from Colombia," Documentos CEDE Series 161356, Universidad de Los Andes, Economics Department.
    18. Ho, Kate & Rosen, Adam M., 2015. "Partial Identification in Applied Research: Benefits and Challenges," CEPR Discussion Papers 10883, C.E.P.R. Discussion Papers.
    19. Claudia Olivetti & Barbara Petrongolo, 2008. "Unequal Pay or Unequal Employment? A Cross-Country Analysis of Gender Gaps," Journal of Labor Economics, University of Chicago Press, vol. 26(4), pages 621-654, October.
    20. Marcel Das & Vera Toepoel & Arthur van Soest, 2011. "Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys," Sociological Methods & Research, , vol. 40(1), pages 32-56, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:statpp:v:3:y:2012:i:1:p:25:n:1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.