IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v57y2023i2d10.1007_s11135-023-01613-y.html
   My bibliography  Save this article

A Rank Graduation Accuracy measure to mitigate Artificial Intelligence risks

Author

Listed:
  • Emanuela Raffinetti

    (University of Pavia)

Abstract

A key point to assess the applications of machine learning models in Artificial Intelligence (AI) is the evaluation of their predictive accuracy. This because the “automatic” choice of an action crucially depends on the made prediction. While the best model in terms of fit to the observed data can be chosen using a “universal” - and therefore automatable - criterion, based on the models’ likelihood, such as AIC and BIC, this is not the case for the best model in terms of predictive accuracy. To fill the gap, we propose a Rank Graduation Accuracy (RGA) measure which evaluates the concordance between the ranks of the predicted values and the ranks of the actual values of a series of observations to be predicted. We apply the RGA to a use-case that concerns the measurement of the financial risks that arise from crypto assets. The RGA appears as a “universal” alternative predictive model selection criterion that, differently from standard measures, such as the Root Mean Squared Error, is robust to the presence of outlying observations.

Suggested Citation

  • Emanuela Raffinetti, 2023. "A Rank Graduation Accuracy measure to mitigate Artificial Intelligence risks," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(2), pages 131-150, December.
    • Handle: RePEc:spr:qualqt:v:57:y:2023:i:2:d:10.1007_s11135-023-01613-y
    DOI: 10.1007/s11135-023-01613-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-023-01613-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-023-01613-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Martin Eling & Davide Nuessle & Julian Staubli, 2022. "The impact of artificial intelligence along the insurance value chain and on the insurability of risks," The Geneva Papers on Risk and Insurance - Issues and Practice, Palgrave Macmillan;The Geneva Association, vol. 47(2), pages 205-241, April.
    3. Angerer, Martin & Hoffmann, Christian Hugo & Neitzert, Florian & Kraus, Sascha, 2021. "Objective and subjective risks of investing into cryptocurrencies," Finance Research Letters, Elsevier, vol. 40(C).
    4. Lerman, Robert I. & Yitzhaki, Shlomo, 1984. "A note on the calculation and interpretation of the Gini index," Economics Letters, Elsevier, vol. 15(3-4), pages 363-368.
    5. Giudici, P. & Raffinetti, E., 2011. "On the Gini measure decomposition," Statistics & Probability Letters, Elsevier, vol. 81(1), pages 133-139, January.
    6. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    7. Paolo Giudici & Emanuela Raffinetti, 2020. "Lorenz Model Selection," Journal of Classification, Springer;The Classification Society, vol. 37(3), pages 754-768, October.
    8. Aldasoro, Iñaki & Gambacorta, Leonardo & Giudici, Paolo & Leach, Thomas, 2022. "The drivers of cyber risk," Journal of Financial Stability, Elsevier, vol. 60(C).
    9. Wenjun Feng & Yiming Wang & Zhengjun Zhang, 2018. "Can cryptocurrencies be a safe haven: a tail risk perspective analysis," Applied Economics, Taylor & Francis Journals, vol. 50(44), pages 4745-4762, September.
    10. Paolo Giudici & Emanuela Raffinetti, 2021. "Cyber risk ordering with rank-based statistical models," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 105(3), pages 469-484, September.
    11. Tae-Ho Kang & Ashish Sharma & Lucy Marshall, 2021. "Assessing Goodness of Fit for Verifying Probabilistic Forecasts," Forecasting, MDPI, vol. 3(4), pages 1-11, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Diego Zappa & Gian Paolo Clemente & Francesco Della Corte & Nino Savelli, 2023. "Editorial on the Special Issue on Insurance: complexity, risks and its connection with social sciences," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(2), pages 125-130, December.
    2. Paolo Giudici & Emanuela Raffinetti, 2025. "RGA: a unified measure of predictive accuracy," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 19(1), pages 67-93, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matteo Malavasi & Gareth W. Peters & Pavel V. Shevchenko & Stefan Truck & Jiwook Jang & Georgy Sofronov, 2021. "Cyber Risk Frequency, Severity and Insurance Viability," Papers 2111.03366, arXiv.org, revised Mar 2022.
    2. Simon Hirsch & Jonathan Berrisch & Florian Ziel, 2024. "Online Distributional Regression," Papers 2407.08750, arXiv.org, revised Aug 2024.
    3. Yousaf, Imran & Yarovaya, Larisa, 2022. "Static and dynamic connectedness between NFTs, Defi and other assets: Portfolio implication," Global Finance Journal, Elsevier, vol. 53(C).
    4. Giudici, Paolo & Gramegna, Alex & Raffinetti, Emanuela, 2023. "Machine Learning Classification Model Comparison," Socio-Economic Planning Sciences, Elsevier, vol. 87(PB).
    5. Paolo Giudici & Emanuela Raffinetti, 2025. "RGA: a unified measure of predictive accuracy," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 19(1), pages 67-93, March.
    6. Jonathan Berrisch & Florian Ziel, 2022. "Distributional modeling and forecasting of natural gas prices," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(6), pages 1065-1086, September.
    7. Diego Zappa & Gian Paolo Clemente & Francesco Della Corte & Nino Savelli, 2023. "Editorial on the Special Issue on Insurance: complexity, risks and its connection with social sciences," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(2), pages 125-130, December.
    8. Paolo Giudici & Emanuela Raffinetti, 2020. "Lorenz Model Selection," Journal of Classification, Springer;The Classification Society, vol. 37(3), pages 754-768, October.
    9. Enilov, Martin & Mensi, Walid & Stankov, Petar, 2023. "Does safe haven exist? Tail risks of commodity markets during COVID-19 pandemic," Journal of Commodity Markets, Elsevier, vol. 29(C).
    10. Mercik, Aleksander & Słoński, Tomasz & Karaś, Marta, 2024. "Understanding crypto-asset exposure: An investigation of its impact on performance and stock sensitivity among listed companies," International Review of Financial Analysis, Elsevier, vol. 92(C).
    11. Iñaki Aldasoro & Leonardo Gambacorta & Paolo Giudici & Thomas Leach, 2023. "Operational and Cyber Risks in the Financial Sector," International Journal of Central Banking, International Journal of Central Banking, vol. 19(5), pages 340-402, December.
    12. Nie, Yan & Zhang, Guoxing & Zhong, Luhao & Su, Bin & Xi, Xi, 2024. "Urban‒rural disparities in household energy and electricity consumption under the influence of electricity price reform policies," Energy Policy, Elsevier, vol. 184(C).
    13. Giudici, Paolo & Piergallini, Alessandro & Recchioni, Maria Cristina & Raffinetti, Emanuela, 2024. "Explainable Artificial Intelligence methods for financial time series," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 655(C).
    14. Quentin Wodon, 2000. "Microdeterminants of consumption, poverty, growth, and inequality in Bangladesh," Applied Economics, Taylor & Francis Journals, vol. 32(10), pages 1337-1352.
    15. Julia Eichholz & Thorsten Knauer & Sandra Winkelmann, 2023. "Digital Maturity of Forecasting and its Impact in Times of Crisis," Schmalenbach Journal of Business Research, Springer, vol. 75(4), pages 443-481, December.
    16. Agarwal, Shivam & Muckley, Cal B. & Neelakantan, Parvati, 2023. "Countering racial discrimination in algorithmic lending: A case for model-agnostic interpretation methods," Economics Letters, Elsevier, vol. 226(C).
    17. Jonathan Berrisch & Florian Ziel, 2023. "Multivariate Probabilistic CRPS Learning with an Application to Day-Ahead Electricity Prices," Papers 2303.10019, arXiv.org, revised Feb 2024.
    18. José Lorenzo, 2002. "E-Index for measuring concentration," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 8(4), pages 357-361, November.
    19. Said Khalil, 2022. "Expectile-based capital allocation," Working Papers hal-03816525, HAL.
    20. Timo Dimitriadis & iaochun Liu & Julie Schnaitmann, 2023. "Encompassing Tests for Value at Risk and Expected Shortfall Multistep Forecasts Based on Inference on the Boundary," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 412-444.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:57:y:2023:i:2:d:10.1007_s11135-023-01613-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.