IDEAS home Printed from https://ideas.repec.org/a/vrs/stintr/v20y2019i4p33-58n11.html
   My bibliography  Save this article

Hybrid Multiple Imputation In A Large Scale Complex Survey

Author

Listed:
  • Razzak Humera

    (Department of Statistics University of Munich, Munich, Germany .)

  • Heumann Christian

    (Department of Statistics University of Munich, Munich, Germany .)

Abstract

Large-scale complex surveys typically contain a large number of variables measured on an even larger number of respondents. Missing data is a common problem in such surveys. Since usually most of the variables in a survey are categorical, multiple imputation requires robust methods for modelling high-dimensional categorical data distributions. This paper introduces the 3-stage Hybrid Multiple Imputation (HMI) approach, computationally efficient and easy to implement, to impute complex survey data sets that contain both continuous and categorical variables. The proposed HMI approach involves the application of sequential regression MI techniques to impute the continuous variables by using information from the categorical variables, already imputed by a non-parametric Bayesian MI approach. The proposed approach seems to be a good alternative to the existing approaches, frequently yielding lower root mean square errors, empirical standard errors and standard errors than the others. The HMI method has proven to be markedly superior to the existing MI methods in terms of computational efficiency. The authors illustrate repeated sampling properties of the hybrid approach using simulated data. The results are also illustrated by child data from the multiple indicator survey (MICS) in Punjab 2014.

Suggested Citation

  • Razzak Humera & Heumann Christian, 2019. "Hybrid Multiple Imputation In A Large Scale Complex Survey," Statistics in Transition New Series, Polish Statistical Association, vol. 20(4), pages 33-58, December.
  • Handle: RePEc:vrs:stintr:v:20:y:2019:i:4:p:33-58:n:11
    DOI: 10.21307/stattrans-2019-033
    as

    Download full text from publisher

    File URL: https://doi.org/10.21307/stattrans-2019-033
    Download Restriction: no

    File URL: https://libkey.io/10.21307/stattrans-2019-033?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. van Buuren, Stef & Groothuis-Oudshoorn, Karin, 2011. "mice: Multivariate Imputation by Chained Equations in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 45(i03).
    2. Olanrewaju Akande & Fan Li & Jerome Reiter, 2017. "An Empirical Comparison of Multiple Imputation Methods for Categorical Data," The American Statistician, Taylor & Francis Journals, vol. 71(2), pages 162-170, April.
    3. Doove, L.L. & Van Buuren, S. & Dusseldorp, E., 2014. "Recursive partitioning for missing data imputation in the presence of interaction effects," Computational Statistics & Data Analysis, Elsevier, vol. 72(C), pages 92-104.
    4. Jared S. Murray & Jerome P. Reiter, 2016. "Multiple Imputation of Missing Categorical and Continuous Values via Bayesian Mixture Models With Local Dependence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1466-1479, October.
    5. Keisuke Hirano, 2002. "Semiparametric Bayesian Inference in Autoregressive Panel Data Models," Econometrica, Econometric Society, vol. 70(2), pages 781-799, March.
    6. Eisele, Martin & Zhu, Junyi, 2013. "Multiple imputation in a complex household survey - the German Panel on Household Finances (PHF): challenges and solutions," EconStor Preprints 100007, ZBW - Leibniz Information Centre for Economics.
    7. Nonyane Bareng A. S. & Foulkes Andrea S., 2007. "Multiple Imputation and Random Forests (MIRF) for Unobservable, High-Dimensional Data," The International Journal of Biostatistics, De Gruyter, vol. 3(1), pages 1-19, August.
    8. Chib, Siddhartha & Hamilton, Barton H., 2002. "Semiparametric Bayes analysis of longitudinal data treatment models," Journal of Econometrics, Elsevier, vol. 110(1), pages 67-89, September.
    9. Horton N.J. & Lipsitz S.R. & Parzen M., 2003. "A Potential for Bias When Rounding in Multiple Imputation," The American Statistician, American Statistical Association, vol. 57, pages 229-232, November.
    10. Iacus, Stefano M. & Porro, Giuseppe, 2007. "Missing data imputation, matching and other applications of random recursive partitioning," Computational Statistics & Data Analysis, Elsevier, vol. 52(2), pages 773-789, October.
    11. Reiter, Jerome P. & Raghunathan, Trivellore E., 2007. "The Multiple Adaptations of Multiple Imputation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1462-1471, December.
    12. Reiter, Jerome P. & Drechsler, Jörg, 2007. "Releasing multiply-imputed synthetic data generated in two stages to protect confidentiality," IAB-Discussion Paper 200720, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    13. Dunson, David B. & Xing, Chuanhua, 2009. "Nonparametric Bayes Modeling of Multivariate Categorical Data," Journal of the American Statistical Association, American Statistical Association, vol. 104(487), pages 1042-1051.
    14. Iacus, Stefano & Porro, Giuseppe, 2008. "Invariant and Metric Free Proximities for Data Matching: An R Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 25(i11).
    15. Su, Yu-Sung & Gelman, Andrew & Hill, Jennifer & Yajima, Masanao, 2011. "Multiple Imputation with Diagnostics (mi) in R: Opening Windows into the Black Box," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 45(i02).
    16. Osamah Basheer Shukur & Muhammad Hisyam Lee, 2015. "Imputation of Missing Values in Daily Wind Speed Data Using Hybrid AR-ANN Method," Modern Applied Science, Canadian Center of Science and Education, vol. 9(11), pages 1-1, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Humera Razzak & Christian Heumann, 2019. "Hybrid Multiple Imputation In A Large Scale Complex Survey," Statistics in Transition New Series, Polish Statistical Association, vol. 20(4), pages 33-58, December.
    2. Yajuan Si & Jerome P. Reiter, 2013. "Nonparametric Bayesian Multiple Imputation for Incomplete Categorical Variables in Large-Scale Assessment Surveys," Journal of Educational and Behavioral Statistics, , vol. 38(5), pages 499-521, October.
    3. Doove, L.L. & Van Buuren, S. & Dusseldorp, E., 2014. "Recursive partitioning for missing data imputation in the presence of interaction effects," Computational Statistics & Data Analysis, Elsevier, vol. 72(C), pages 92-104.
    4. Rashid, S. & Mitra, R. & Steele, R.J., 2015. "Using mixtures of t densities to make inferences in the presence of missing data with a small number of multiply imputed data sets," Computational Statistics & Data Analysis, Elsevier, vol. 92(C), pages 84-96.
    5. Speidel, Matthias & Drechsler, Jörg & Jolani, Shahab, 2018. "R package hmi: a convenient tool for hierarchical multiple imputation and beyond," IAB-Discussion Paper 201816, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    6. Jensen, Mark J. & Maheu, John M., 2010. "Bayesian semiparametric stochastic volatility modeling," Journal of Econometrics, Elsevier, vol. 157(2), pages 306-316, August.
    7. Svetlana Zhuchkova & Aleksei Rotmistrov, 2022. "How to choose an approach to handling missing categorical data: (un)expected findings from a simulated statistical experiment," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(1), pages 1-22, February.
    8. Laura Liu & Hyungsik Roger Moon & Frank Schorfheide, 2023. "Forecasting with a panel Tobit model," Quantitative Economics, Econometric Society, vol. 14(1), pages 117-159, January.
    9. Drechsler, Jörg & Reiter, Jerome P., 2011. "An empirical evaluation of easily implemented, nonparametric methods for generating synthetic datasets," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3232-3243, December.
    10. Fisher, Mark & Jensen, Mark J., 2022. "Bayesian nonparametric learning of how skill is distributed across the mutual fund industry," Journal of Econometrics, Elsevier, vol. 230(1), pages 131-153.
    11. Gerko Vink & Laurence E. Frank & Jeroen Pannekoek & Stef Buuren, 2014. "Predictive mean matching imputation of semicontinuous variables," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 68(1), pages 61-90, February.
    12. Federico Bassetti & Roberto Casarin & Marco Del Negro, 2022. "A Bayesian Approach to Inference on Probabilistic Surveys," Staff Reports 1025, Federal Reserve Bank of New York.
    13. Abel Rodriguez & Enrique ter Horst, 2008. "Measuring expectations in options markets: An application to the SP500 index," Papers 0901.0033, arXiv.org.
    14. Ryo Kato & Takahiro Hoshino, 2020. "Semiparametric Bayesian multiple imputation for regression models with missing mixed continuous–discrete covariates," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(3), pages 803-825, June.
    15. Youngjoo Cho & Debashis Ghosh, 2021. "Quantile-Based Subgroup Identification for Randomized Clinical Trials," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 13(1), pages 90-128, April.
    16. repec:cup:judgdm:v:15:y:2020:i:5:p:798-806 is not listed on IDEAS
    17. Billio, Monica & Casarin, Roberto & Rossini, Luca, 2019. "Bayesian nonparametric sparse VAR models," Journal of Econometrics, Elsevier, vol. 212(1), pages 97-115.
    18. Federico Bassetti & Roberto Casarin & Francesco Ravazzolo, 2018. "Bayesian Nonparametric Calibration and Combination of Predictive Distributions," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 675-685, April.
    19. Griffin, J. E. & Steel, M. F. J., 2004. "Semiparametric Bayesian inference for stochastic frontier models," Journal of Econometrics, Elsevier, vol. 123(1), pages 121-152, November.
    20. Stuart R. Lipsitz & Garrett M. Fitzmaurice & Roger D. Weiss, 2020. "Using Multiple Imputation with GEE with Non-monotone Missing Longitudinal Binary Outcomes," Psychometrika, Springer;The Psychometric Society, vol. 85(4), pages 890-904, December.
    21. Fisher, Mark & Jensen, Mark J., 2019. "Bayesian inference and prediction of a multiple-change-point panel model with nonparametric priors," Journal of Econometrics, Elsevier, vol. 210(1), pages 187-202.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vrs:stintr:v:20:y:2019:i:4:p:33-58:n:11. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.sciendo.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.