IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i2p438-451.html
   My bibliography  Save this article

On the generative-discriminative tradeoff approach: Interpretation, asymptotic efficiency and classification performance

Author

Listed:
  • Xue, Jing-Hao
  • Titterington, D. Michael

Abstract

The interpretation of generative, discriminative and hybrid approaches to classification is discussed, in particular for the generative-discriminative tradeoff (GDT), a hybrid approach. The asymptotic efficiency of the GDT, relative to that of its generative or discriminative counterpart, is presented theoretically and, by using linear normal discrimination as an example, numerically. On real and simulated datasets, the classification performance of the GDT is compared with those of normal-based linear discriminant analysis (LDA) and linear logistic regression (LLR). Four arguments are made as follows. First, the GDT is a generative model integrating both discriminative and generative learning. It is therefore subject to model misspecification of the data-generating process and hindered by complex optimisation. Secondly, among the three approaches being compared, the asymptotic efficiency of the GDT is higher than that of the discriminative approach but lower than that of the generative approach, when no model misspecification occurs. Thirdly, without model misspecification, LDA performs the best; with model misspecification, LLR or the GDT with an optimal, large weight on its discriminative component may perform the best. Finally, LLR is affected by the imbalance between groups of data.

Suggested Citation

  • Xue, Jing-Hao & Titterington, D. Michael, 2010. "On the generative-discriminative tradeoff approach: Interpretation, asymptotic efficiency and classification performance," Computational Statistics & Data Analysis, Elsevier, vol. 54(2), pages 438-451, February.
  • Handle: RePEc:eee:csdana:v:54:y:2010:i:2:p:438-451
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(09)00327-2
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kim, Ji-Hyun, 2009. "Estimating classification error rate: Repeated cross-validation, repeated hold-out and bootstrap," Computational Statistics & Data Analysis, Elsevier, vol. 53(11), pages 3735-3745, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mark G E White & Neil E Bezodis & Jonathon Neville & Huw Summers & Paul Rees, 2022. "Determining jumping performance from a single body-worn accelerometer using machine learning," PLOS ONE, Public Library of Science, vol. 17(2), pages 1-25, February.
    2. Richard A. Johansen & Molly K. Reif & Christina L. Saltus & Kaytee L. Pokrzywinski, 2024. "A Broadscale Assessment of Sentinel-2 Imagery and the Google Earth Engine for the Nationwide Mapping of Chlorophyll a," Sustainability, MDPI, vol. 16(5), pages 1-17, March.
    3. Airola, Antti & Pahikkala, Tapio & Waegeman, Willem & De Baets, Bernard & Salakoski, Tapio, 2011. "An experimental comparison of cross-validation techniques for estimating the area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1828-1844, April.
    4. Matthias Schmid & Thomas Hielscher & Thomas Augustin & Olaf Gefeller, 2011. "A Robust Alternative to the Schemper–Henderson Estimator of Prediction Error," Biometrics, The International Biometric Society, vol. 67(2), pages 524-535, June.
    5. Luts, Jan & Ormerod, John T., 2014. "Mean field variational Bayesian inference for support vector machine classification," Computational Statistics & Data Analysis, Elsevier, vol. 73(C), pages 163-176.
    6. David Rios Insua & Roi Naveiro & Victor Gallego, 2020. "Perspectives on Adversarial Classification," Mathematics, MDPI, vol. 8(11), pages 1-21, November.
    7. John J Nay & Yevgeniy Vorobeychik, 2016. "Predicting Human Cooperation," PLOS ONE, Public Library of Science, vol. 11(5), pages 1-19, May.
    8. Matthew Tuson & Berwin Turlach & Kevin Murray & Mei Ruu Kok & Alistair Vickery & David Whyatt, 2021. "Predicting Future Geographic Hotspots of Potentially Preventable Hospitalisations Using All Subset Model Selection and Repeated K-Fold Cross-Validation," IJERPH, MDPI, vol. 18(19), pages 1-21, September.
    9. Lauri Nevasalmi, 2022. "Recession forecasting with high‐dimensional data," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(4), pages 752-764, July.
    10. Usta, Ilhan & Kantar, Yeliz Mert, 2011. "On the performance of the flexible maximum entropy distributions within partially adaptive estimation," Computational Statistics & Data Analysis, Elsevier, vol. 55(6), pages 2172-2182, June.
    11. Hosseini, Fatemeh & Eidsvik, Jo & Mohammadzadeh, Mohsen, 2011. "Approximate Bayesian inference in spatial GLMM with skew normal latent variables," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1791-1806, April.
    12. Gonzalo Perez-de-la-Cruz & Guillermina Eslava-Gomez, 2019. "Discriminant analysis for discrete variables derived from a tree-structured graphical model," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 13(4), pages 855-876, December.
    13. I. Charvet & A. Suppasri & H. Kimura & D. Sugawara & F. Imamura, 2015. "A multivariate generalized linear tsunami fragility model for Kesennuma City based on maximum flow depths, velocities and debris impact, with evaluation of predictive accuracy," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 79(3), pages 2073-2099, December.
    14. Shusaku Tsumoto & Tomohirno Kimura & Shoji Hirano, 2021. "Determination of Disease from Discharge Summaries," The Review of Socionetwork Strategies, Springer, vol. 15(1), pages 49-66, June.
    15. Khan, Jafar A. & Van Aelst, Stefan & Zamar, Ruben H., 2010. "Fast robust estimation of prediction error based on resampling," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3121-3130, December.
    16. Chen, Weijie & Yousef, Waleed A. & Gallas, Brandon D. & Hsu, Elizabeth R. & Lababidi, Samir & Tang, Rong & Pennello, Gene A. & Symmans, W. Fraser & Pusztai, Lajos, 2012. "Uncertainty estimation with a finite dataset in the assessment of classification models," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1016-1027.
    17. Mark Lown & Michael Brown & Chloë Brown & Arthur M Yue & Benoy N Shah & Simon J Corbett & George Lewith & Beth Stuart & Michael Moore & Paul Little, 2020. "Machine learning detection of Atrial Fibrillation using wearable technology," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-9, January.
    18. Piccarreta, Raffaella, 2010. "Binary trees for dissimilarity data," Computational Statistics & Data Analysis, Elsevier, vol. 54(6), pages 1516-1524, June.
    19. Ha, Tran Vinh & Asada, Takumi & Arimura, Mikiharu, 2019. "Determination of the influence factors on household vehicle ownership patterns in Phnom Penh using statistical and machine learning methods," Journal of Transport Geography, Elsevier, vol. 78(C), pages 70-86.
    20. Charlotte Soneson & Sarah Gerster & Mauro Delorenzi, 2014. "Batch Effect Confounding Leads to Strong Bias in Performance Estimates Obtained by Cross-Validation," PLOS ONE, Public Library of Science, vol. 9(6), pages 1-13, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:2:p:438-451. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.