IDEAS home Printed from https://ideas.repec.org/a/spr/advdac/v11y2017i2d10.1007_s11634-016-0243-0.html
   My bibliography  Save this article

A sequential distance-based approach for imputing missing data: Forward Imputation

Author

Listed:
  • Nadia Solaro

    (Università degli Studi di Milano-Bicocca)

  • Alessandro Barbiero

    (Università degli Studi di Milano)

  • Giancarlo Manzi

    (Università degli Studi di Milano)

  • Pier Alda Ferrari

    (Università degli Studi di Milano)

Abstract

Missing data recurrently affect datasets in almost every field of quantitative research. The subject is vast and complex and has originated a literature rich in very different approaches to the problem. Within an exploratory framework, distance-based methods such as nearest-neighbour imputation (NNI), or procedures involving multivariate data analysis (MVDA) techniques seem to treat the problem properly. In NNI, the metric and the number of donors can be chosen at will. MVDA-based procedures expressly account for variable associations. The new approach proposed here, called Forward Imputation, ideally meets these features. It is designed as a sequential procedure that imputes missing data in a step-by-step process involving subsets of units according to their “completeness rate”. Two methods within this context are developed for the imputation of quantitative data. One applies NNI with the Mahalanobis distance, the other combines NNI and principal component analysis. Statistical properties of the two methods are discussed, and their performance is assessed, also in comparison with alternative imputation methods. To this purpose, a simulation study in the presence of different data patterns along with an application to real data are carried out, and practical hints for users are also provided.

Suggested Citation

  • Nadia Solaro & Alessandro Barbiero & Giancarlo Manzi & Pier Alda Ferrari, 2017. "A sequential distance-based approach for imputing missing data: Forward Imputation," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 11(2), pages 395-414, June.
    • Handle: RePEc:spr:advdac:v:11:y:2017:i:2:d:10.1007_s11634-016-0243-0
    DOI: 10.1007/s11634-016-0243-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11634-016-0243-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11634-016-0243-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Agostino Tarsitano & Marianna Falcone, 2010. "Missing-Values Adjustment For Mixed-Type Data," Working Papers 201015, Università della Calabria, Dipartimento di Economia, Statistica e Finanza "Giovanni Anania" - DESF.
    2. A. Azzalini & A. Capitanio, 1999. "Statistical applications of the multivariate skew normal distribution," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(3), pages 579-602.
    3. Nadia SOLARO & Alessandro BARBIERO & Giancarlo MANZI & Pier Alda FERRARI, 2015. "A Comprehensive Simulation Study on the Forward Imputation," Departmental Working Papers 2015-04, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    4. Lounasheimo, Antton, 1999. "The Impact of Human Capital on Economic Growth," Discussion Papers 673, The Research Institute of the Finnish Economy.
    5. Julie Josse & Jérôme Pagès & François Husson, 2011. "Multiple imputation in principal component analysis," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 5(3), pages 231-246, October.
    6. Ferrari, Pier Alda & Annoni, Paola & Barbiero, Alessandro & Manzi, Giancarlo, 2011. "An imputation method for categorical variables with application to nonlinear principal component analysis," Computational Statistics & Data Analysis, Elsevier, vol. 55(7), pages 2410-2420, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Emanuela Raffinetti & Pier Alda Ferrari, 2021. "A dependence measure flow tree through Monte Carlo simulations," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(2), pages 467-496, April.
    2. Atieno, Prisca, 2021. "The effects of outdated data and outliers on Kenya's 2019 Global Food Security Index score and rank," Research Theses 334773, Collaborative Masters Program in Agricultural and Applied Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anna Gottard & Simona Pacillo, 2007. "On the impact of contaminations in graphical Gaussian models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 15(3), pages 343-354, February.
    2. David Mayston, 2015. "Analysing the effectiveness of public service producers with endogenous resourcing," Journal of Productivity Analysis, Springer, vol. 44(1), pages 115-126, August.
    3. Camila Zeller & Victor Lachos & Filidor Labra, 2014. "Influence diagnostics for Grubbs’s model with asymmetric heavy-tailed distributions," Statistical Papers, Springer, vol. 55(3), pages 671-690, August.
    4. Wan-Lun Wang & Min Liu & Tsung-I Lin, 2017. "Robust skew-t factor analysis models for handling missing data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 26(4), pages 649-672, November.
    5. Olcay Arslan, 2015. "Variance-mean mixture of the multivariate skew normal distribution," Statistical Papers, Springer, vol. 56(2), pages 353-378, May.
    6. Mehdi Amiri & Ahad Jamalizadeh & Mina Towhidi, 2015. "Inference and further probabilistic properties of the $$ SUN_{n,2}$$ S U N n , 2 -distribution," Statistical Papers, Springer, vol. 56(4), pages 1071-1098, November.
    7. Jaakko Nevalainen & Denis Larocque & Hannu Oja, 2007. "A weighted spatial median for clustered data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 15(3), pages 355-379, February.
    8. Hea-Jung Kim, 2015. "A best linear threshold classification with scale mixture of skew normal populations," Computational Statistics, Springer, vol. 30(1), pages 1-28, March.
    9. Tsung-I Lin & Pal Wu & Geoffrey McLachlan & Sharon Lee, 2015. "A robust factor analysis model using the restricted skew- $$t$$ t distribution," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(3), pages 510-531, September.
    10. Wan-Lun Wang & Tsung-I Lin, 2015. "Robust model-based clustering via mixtures of skew-t distributions with missing information," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 9(4), pages 423-445, December.
    11. Padilla, Juan L. & Azevedo, Caio L.N. & Lachos, Victor H., 2018. "Multidimensional multiple group IRT models with skew normal latent trait distributions," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 250-268.
    12. Marco Minozzo & Luca Bagnato, 2021. "A unified skew‐normal geostatistical factor model," Environmetrics, John Wiley & Sons, Ltd., vol. 32(4), June.
    13. Bernardi, Mauro, 2013. "Risk measures for skew normal mixtures," Statistics & Probability Letters, Elsevier, vol. 83(8), pages 1819-1824.
    14. Panagiotelis, Anastasios & Smith, Michael, 2010. "Bayesian skew selection for multivariate models," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1824-1839, July.
    15. Katherine Elizabeth Castellano & Andrew Dean Ho, 2013. "Contrasting OLS and Quantile Regression Approaches to Student “Growth†Percentiles," Journal of Educational and Behavioral Statistics, , vol. 38(2), pages 190-215, April.
    16. Mitchell, James & Weale, Martin, 2019. "Forecasting with Unknown Unknowns: Censoring and Fat Tails on the Bank of England's Monetary Policy Committee," EMF Research Papers 27, Economic Modelling and Forecasting Group.
    17. Reinaldo B. Arellano-Valle & Marc G. Genton, 2010. "Multivariate extended skew-t distributions and related families," Metron - International Journal of Statistics, Dipartimento di Statistica, Probabilità e Statistiche Applicate - University of Rome, vol. 0(3), pages 201-234.
    18. J. T. A. S. Ferreira & M. F. J. Steel, 2004. "On Describing Multivariate Skewness: A Directional Approach," Econometrics 0409010, University Library of Munich, Germany.
    19. Lachos, Victor H. & Prates, Marcos O. & Dey, Dipak K., 2021. "Heckman selection-t model: Parameter estimation via the EM-algorithm," Journal of Multivariate Analysis, Elsevier, vol. 184(C).
    20. M. Teimourian & T. Baghfalaki & M. Ganjali & D. Berridge, 2015. "Joint modeling of mixed skewed continuous and ordinal longitudinal responses: a Bayesian approach," Journal of Applied Statistics, Taylor & Francis Journals, vol. 42(10), pages 2233-2256, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:advdac:v:11:y:2017:i:2:d:10.1007_s11634-016-0243-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.