IDEAS home Printed from https://ideas.repec.org/a/taf/japsta/v41y2014i8p1678-1693.html
   My bibliography  Save this article

Optimal cut-off for rare events and unbalanced misclassification costs

Author

Listed:
  • Raffaella Calabrese

Abstract

This paper develops a method for handling two-class classification problems with highly unbalanced class sizes and misclassification costs. When the class sizes are highly unbalanced and the minority class represents a rare event, conventional classification methods tend to strongly favour the majority class, resulting in very low detection of the minority class. A method is proposed to determine the optimal cut-off for asymmetric misclassification costs and for unbalanced class sizes. Monte Carlo simulations show that this proposal performs better than the method based on the notion of classification accuracy. Finally, the proposed method is applied to empirical data on Italian small and medium enterprises to classify them into default and non-default groups.

Suggested Citation

  • Raffaella Calabrese, 2014. "Optimal cut-off for rare events and unbalanced misclassification costs," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(8), pages 1678-1693, August.
    • Handle: RePEc:taf:japsta:v:41:y:2014:i:8:p:1678-1693
    DOI: 10.1080/02664763.2014.888542
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/02664763.2014.888542
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/02664763.2014.888542?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Barry C. Arnold, 2008. "Pareto and Generalized Pareto Distributions," Economic Studies in Inequality, Social Exclusion, and Well-Being, in: Duangkamon Chotikapanich (ed.), Modeling Income Distributions and Lorenz Curves, chapter 7, pages 119-145, Springer.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. D. Bidzhoyan S. & Д. Биджоян С., 2018. "Модель Оценки Вероятности Отзыва Лицензии У Российского Банка // Model For Assessing The Probability Of Revocation Of A License From The Russian Bank," Финансы: теория и практика/Finance: Theory and Practice // Finance: Theory and Practice, ФГОБУВО Финансовый университет при Правительстве Российской Федерации // Financial University under The Government of Russian Federation, vol. 22(2), pages 26-37.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuanyuan Zhang & Saralees Nadarajah, 2017. "Flexible Heavy Tailed Distributions for Big Data," Annals of Data Science, Springer, vol. 4(3), pages 421-432, September.
    2. Majda Benzidia & Michel Lubrano, 2016. "A Bayesian Look at American Academic Wages: The Case of Michigan State University," AMSE Working Papers 1628, Aix-Marseille School of Economics, France.
    3. Matthew Fisher-Post, 2020. "Examining the Great Leveling: New Evidence on Midcentury American Inequality," PSE Working Papers hal-02876981, HAL.
    4. Vladimir Hlasny, 2019. "Redistributive Impacts of Fiscal Policies in Mexico: Corrections for Top Income Measurement Problems," LIS Working papers 765, LIS Cross-National Data Center in Luxembourg.
    5. Faustino Prieto & José María Sarabia & Enrique Calderín-Ojeda, 2021. "The nonlinear distribution of employment across municipalities," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 16(2), pages 287-307, April.
    6. Alberto Arcagni & Francesco Porro, 2013. "On the parameters of Zenga distribution," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 22(3), pages 285-303, August.
    7. Vladimir Hlasny, 2021. "Parametric representation of the top of income distributions: Options, historical evidence, and model selection," Journal of Economic Surveys, Wiley Blackwell, vol. 35(4), pages 1217-1256, September.
    8. Reto Foellmi & Isabel Z. Martínez, 2017. "Volatile Top Income Shares in Switzerland? Reassessing the Evolution between 1981 and 2010," The Review of Economics and Statistics, MIT Press, vol. 99(5), pages 793-809, December.
    9. Enora Belz, 2019. "Estimating Inequality Measures from Quantile Data," Working Papers halshs-02320110, HAL.
    10. Frank A. Cowell & Philippe Kerm, 2015. "Wealth Inequality: A Survey," Journal of Economic Surveys, Wiley Blackwell, vol. 29(4), pages 671-710, September.
    11. Stephen P. Jenkins, 2017. "Pareto Models, Top Incomes and Recent Trends in UK Income Inequality," Economica, London School of Economics and Political Science, vol. 84(334), pages 261-289, April.
    12. Vladimir Hlasny & Paolo Verme, 2022. "The Impact of Top Incomes Biases on the Measurement of Inequality in the United States," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 84(4), pages 749-788, August.
    13. Foellmi, Reto & Martínez, Isabel Z., 2014. "Volatile Top Income Shares in Switzerland? Reassessing the Evolution Between 1981 and 2009," CEPR Discussion Papers 10006, C.E.P.R. Discussion Papers.
    14. Wang, Yuanjun & You, Shibing, 2016. "An alternative method for modeling the size distribution of top wealth," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 457(C), pages 443-453.
    15. Omer L. Gebizlioglu & Serap Yörübulut, 2016. "A Pseudo-Pareto Distribution and Concomitants of Its Order Statistics," Methodology and Computing in Applied Probability, Springer, vol. 18(4), pages 1043-1064, December.
    16. Nadav Ben Zeev & Tomer Ifergane, 2022. "Firing Restrictions and Economic Resilience: Protect and Survive?," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 43, pages 93-124, January.
    17. Oikonomou, Thomas & Kaloudis, Konstantinos & Bagci, G. Baris, 2021. "The q-exponentials do not maximize the Rényi entropy," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 578(C).
    18. Enora Belz, 2019. "Estimating Inequality Measures from Quantile Data," Economics Working Paper Archive (University of Rennes & University of Caen) 2019-09, Center for Research in Economics and Management (CREM), University of Rennes, University of Caen and CNRS.
    19. repec:exl:29stat:v:20:y:2019:i:3:p:57-80 is not listed on IDEAS
    20. Matthew Fisher-Post, 2020. "Examining the Great Leveling: New Evidence on Midcentury American Inequality," World Inequality Lab Working Papers hal-02876981, HAL.
    21. Muhammad Hilmi Abdul Majid & Kamarulzaman Ibrahim, 2021. "On Bayesian approach to composite Pareto models," PLOS ONE, Public Library of Science, vol. 16(9), pages 1-22, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:japsta:v:41:y:2014:i:8:p:1678-1693. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/CJAS20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.