IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v179y2023ics0167947322002080.html
   My bibliography  Save this article

Tree-based ensembles for multi-output regression: Comparing multivariate approaches with separate univariate ones

Author

Listed:
  • Schmid, Lena
  • Gerharz, Alexander
  • Groll, Andreas
  • Pauly, Markus

Abstract

Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. In particular, they are used for predicting univariate responses. In case of multiple outputs the question arises whether it is better to separately fit univariate models or directly follow a multivariate approach. For the latter, several possibilities exist that are, e.g. based on modified splitting or stopping rules for multi-output regression. These methods are compared in extensive simulations and a real data example to help in answering the primary question when to use multivariate ensemble techniques instead of univariate ones.

Suggested Citation

  • Schmid, Lena & Gerharz, Alexander & Groll, Andreas & Pauly, Markus, 2023. "Tree-based ensembles for multi-output regression: Comparing multivariate approaches with separate univariate ones," Computational Statistics & Data Analysis, Elsevier, vol. 179(C).
    • Handle: RePEc:eee:csdana:v:179:y:2023:i:c:s0167947322002080
    DOI: 10.1016/j.csda.2022.107628
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947322002080
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2022.107628?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Wright, Marvin N. & Ziegler, Andreas, 2017. "ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 77(i01).
    2. David R. Larsen & Paul L. Speckman, 2004. "Multivariate Regression Trees for Analysis of Abundance Data," Biometrics, The International Biometric Society, vol. 60(2), pages 543-549, June.
    3. Zimmermann, Georg & Pauly, Markus & Bathke, Arne C., 2020. "Multivariate analysis of covariance with potentially singular covariance matrices and non-normal responses," Journal of Multivariate Analysis, Elsevier, vol. 177(C).
    4. Dennis Dobler & Sarah Friedrich & Markus Pauly, 2020. "Nonparametric MANOVA in meaningful effects," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(4), pages 997-1022, August.
    5. Véronique Cariou, 2006. "Extension of multivariate regression trees to interval data. Application to electricity load profiling," Computational Statistics, Springer, vol. 21(2), pages 325-341, June.
    6. Dine, Abdessamad & Larocque, Denis & Bellavance, François, 2009. "Multivariate trees for mixed outcomes," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3795-3804, September.
    7. Siciliano, Roberta & Mola, Francesco, 2000. "Multivariate data analysis and modeling through classification and regression trees," Computational Statistics & Data Analysis, Elsevier, vol. 32(3-4), pages 285-301, January.
    8. Hsiao, Wei-Cheng & Shih, Yu-Shan, 2007. "Splitting variable selection for multivariate regression trees," Statistics & Probability Letters, Elsevier, vol. 77(3), pages 265-271, February.
    9. Keon Lee, Seong, 2005. "On generalized multivariate decision tree by using GEE," Computational Statistics & Data Analysis, Elsevier, vol. 49(4), pages 1105-1119, June.
    10. Konietschke, Frank & Bathke, Arne C. & Harrar, Solomon W. & Pauly, Markus, 2015. "Parametric and nonparametric bootstrap methods for general MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 140(C), pages 291-301.
    11. Kocev, Dragi & Džeroski, Sašo & White, Matt D. & Newell, Graeme R. & Griffioen, Peter, 2009. "Using single- and multi-target regression trees and ensembles to model a compound index of vegetation condition," Ecological Modelling, Elsevier, vol. 220(8), pages 1159-1168.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Federico Zincenko, 2023. "Nonparametric estimation of conditional densities by generalized random forests," Papers 2309.13251, arXiv.org, revised Jan 2024.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Piccarreta, Raffaella, 2010. "Binary trees for dissimilarity data," Computational Statistics & Data Analysis, Elsevier, vol. 54(6), pages 1516-1524, June.
    2. Wei-Yin Loh, 2014. "Fifty Years of Classification and Regression Trees," International Statistical Review, International Statistical Institute, vol. 82(3), pages 329-348, December.
    3. Hsiao, Wei-Cheng & Shih, Yu-Shan, 2007. "Splitting variable selection for multivariate regression trees," Statistics & Probability Letters, Elsevier, vol. 77(3), pages 265-271, February.
    4. Antonio D’Ambrosio & Willem J. Heiser, 2016. "A Recursive Partitioning Method for the Prediction of Preference Rankings Based Upon Kemeny Distances," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 774-794, September.
    5. Dine, Abdessamad & Larocque, Denis & Bellavance, François, 2009. "Multivariate trees for mixed outcomes," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3795-3804, September.
    6. Friedrich, Sarah & Pauly, Markus, 2018. "MATS: Inference for potentially singular and heteroscedastic MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 166-179.
    7. Mariana Oliveira & Luís Torgo & Vítor Santos Costa, 2021. "Evaluation Procedures for Forecasting with Spatiotemporal Data," Mathematics, MDPI, vol. 9(6), pages 1-27, March.
    8. Smaga, Łukasz, 2015. "Wald-type statistics using {2}-inverses for hypothesis testing in general factorial designs," Statistics & Probability Letters, Elsevier, vol. 107(C), pages 215-220.
    9. Dickhaus, Thorsten & Sirotko-Sibirskaya, Natalia, 2019. "Simultaneous statistical inference in dynamic factor models: Chi-square approximation and model-based bootstrap," Computational Statistics & Data Analysis, Elsevier, vol. 129(C), pages 30-46.
    10. Arjan S. Gosal & Janine A. McMahon & Katharine M. Bowgen & Catherine H. Hoppe & Guy Ziv, 2021. "Identifying and Mapping Groups of Protected Area Visitors by Environmental Awareness," Land, MDPI, vol. 10(6), pages 1-14, May.
    11. Albert Stuart Reece & Gary Kenneth Hulse, 2022. "European Epidemiological Patterns of Cannabis- and Substance-Related Congenital Neurological Anomalies: Geospatiotemporal and Causal Inferential Study," IJERPH, MDPI, vol. 20(1), pages 1-35, December.
    12. Michael Parzinger & Lucia Hanfstaengl & Ferdinand Sigg & Uli Spindler & Ulrich Wellisch & Markus Wirnsberger, 2020. "Residual Analysis of Predictive Modelling Data for Automated Fault Detection in Building’s Heating, Ventilation and Air Conditioning Systems," Sustainability, MDPI, vol. 12(17), pages 1-18, August.
    13. Pier Perri & Peter Heijden, 2012. "A Property of the CHAID Partitioning Method for Dichotomous Randomized Response Data and Categorical Predictors," Journal of Classification, Springer;The Classification Society, vol. 29(1), pages 76-90, April.
    14. Van Belle, Jente & Guns, Tias & Verbeke, Wouter, 2021. "Using shared sell-through data to forecast wholesaler demand in multi-echelon supply chains," European Journal of Operational Research, Elsevier, vol. 288(2), pages 466-479.
    15. Albert Stuart Reece & Gary Kenneth Hulse, 2022. "European Epidemiological Patterns of Cannabis- and Substance-Related Body Wall Congenital Anomalies: Geospatiotemporal and Causal Inferential Study," IJERPH, MDPI, vol. 19(15), pages 1-38, July.
    16. Philipp Bach & Victor Chernozhukov & Malte S. Kurz & Martin Spindler & Sven Klaassen, 2021. "DoubleML -- An Object-Oriented Implementation of Double Machine Learning in R," Papers 2103.09603, arXiv.org, revised Feb 2024.
    17. Marchetto, Elisa & Da Re, Daniele & Tordoni, Enrico & Bazzichetto, Manuele & Zannini, Piero & Celebrin, Simone & Chieffallo, Ludovico & Malavasi, Marco & Rocchini, Duccio, 2023. "Testing the effect of sample prevalence and sampling methods on probability- and favourability-based SDMs," Ecological Modelling, Elsevier, vol. 477(C).
    18. Jorge Luis Andrade & José Luis Valencia, 2022. "A Fuzzy Random Survival Forest for Predicting Lapses in Insurance Portfolios Containing Imprecise Data," Mathematics, MDPI, vol. 11(1), pages 1-16, December.
    19. Eeva-Katri Kumpula & Pauline Norris & Adam C Pomerleau, 2020. "Stocks of paracetamol products stored in urban New Zealand households: A cross-sectional study," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-11, June.
    20. Michael Bucker & Gero Szepannek & Alicja Gosiewska & Przemyslaw Biecek, 2020. "Transparency, Auditability and eXplainability of Machine Learning Models in Credit Scoring," Papers 2009.13384, arXiv.org.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:179:y:2023:i:c:s0167947322002080. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.