IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2609.26445.html

A Practical Guide on Graphical Model Validation

Author

Listed:
  • Mario V. Wuthrich

Abstract

This manuscript formalizes the most popular model validation tools used in general insurance actuarial modeling. These include graphical tools like calibration plots, actual-vs-expected plots, lift charts, Murphy diagrams, as well as classical statistical tools such as Bregman losses, deviance losses, elementary losses, Murphy's decomposition and Gini scores. Particular emphasis is placed on whether calibration and discrimination are studied under a policy-weighted or an exposure-weighted population measure. This distinction is crucial in ensuring that premium schemes are calibrated on the correct scale.

Suggested Citation

  • Mario V. Wuthrich, 2026. "A Practical Guide on Graphical Model Validation," Papers 2609.26445, arXiv.org.
    • Handle: RePEc:arx:papers:2609.26445
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2609.26445
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Denuit, Michel & Huyghe, Julie & Simon, Pierre-Alexandre & Trufin, Julien, 2026. "Tweedie dominance for autocalibrated predictors and Laplace transform order," LIDAM Reprints ISBA 2026025, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    2. Michel Denuit & Arthur Charpentier & Julien Trufin, 2021. "Autocalibration and Tweedie-dominance for Insurance Pricing with Machine Learning," Papers 2103.03635, arXiv.org, revised Jul 2021.
    3. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    4. Michel Denuit & Julie Huyghe & Pierre-Alexandre Simon & Julien Trufin, 2026. "Tweedie dominance for autocalibrated predictors and Laplace transform order," Scandinavian Actuarial Journal, Taylor & Francis Journals, vol. 2026(6), pages 569-583, July.
    5. Denuit, Michel & Trufin, Julien, 2021. "Lorenz curve, Gini coefficient, and Tweedie dominance for autocalibrated predictors," LIDAM Discussion Papers ISBA 2021036, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    6. Dimitriadis, Timo & Gneiting, Tilmann & Jordan, Alexander I. & Vogel, Peter, 2024. "Evaluating probabilistic classifiers: The triptych," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1101-1122.
    7. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    8. M. Lindholm & F. Lindskog & J. Palmquist, 2023. "Local bias adjustment, duration-weighted probabilities, and automatic construction of tariff cells," Scandinavian Actuarial Journal, Taylor & Francis Journals, vol. 2023(10), pages 946-973, November.
    9. J. Kruskal, 1964. "Nonmetric multidimensional scaling: A numerical method," Psychometrika, Springer;The Psychometric Society, vol. 29(2), pages 115-129, June.
    10. Alexej Brauer & Paul Menzel & Mario V. Wuthrich, 2025. "Model Monitoring: A General Framework with an Application to Non-life Insurance Pricing," Papers 2510.04556, arXiv.org, revised Dec 2025.
    11. Lindholm, M. & Richman, R. & Tsanakas, A. & Wüthrich, M.V., 2022. "Discrimination-Free Insurance Pricing," ASTIN Bulletin, Cambridge University Press, vol. 52(1), pages 55-89, January.
    12. Denuit, Michel & Charpentier, Arthur & Trufin, Julien, 2021. "Autocalibration and Tweedie-dominance for insurance pricing with machine learning," LIDAM Discussion Papers ISBA 2021013, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    13. Denuit, Michel & Trufin, Julien, 2024. "Convex and Lorenz orders under balance correction in nonlife insurance pricing: Review and new developments," Insurance: Mathematics and Economics, Elsevier, vol. 118(C), pages 123-128.
    14. Denuit, Michel & Huyghe, Julie & Trufin, Julien & Verdebout, Thomas, 2024. "Testing for auto-calibration with Lorenz and Concentration curves," Insurance: Mathematics and Economics, Elsevier, vol. 117(C), pages 130-139.
    15. Denuit, Michel & Charpentier , Arthur & Trufin, Julien, 2021. "Autocalibration and Tweedie-dominance for insurance pricing with machine learning," LIDAM Reprints ISBA 2021049, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    16. Christian Gourieroux & Joann Jasiak, 2007. "Introduction to The Econometrics of Individual Risk: Credit, Insurance, and Marketing," Introductory Chapters, in: The Econometrics of Individual Risk: Credit, Insurance, and Marketing, Princeton University Press.
    17. Denuit, Michel & Trufin, Julien, 2023. "Model selection with Pearson’s correlation, concentration and Lorenz curves under autocalibration," LIDAM Reprints ISBA 2023025, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    18. Denuit, Michel & Sznajder, Dominik & Trufin, Julien, 2019. "Model selection based on Lorenz and concentration curves, Gini indices and convex order," LIDAM Reprints ISBA 2019046, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    19. de Leeuw, Jan & Hornik, Kurt & Mair, Patrick, 2009. "Isotone Optimization in R: Pool-Adjacent-Violators Algorithm (PAVA) and Active Set Methods," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 32(i05).
    20. Denuit, Michel & Charpentier, Arthur & Trufin, Julien, 2021. "Autocalibration and Tweedie-dominance for insurance pricing with machine learning," Insurance: Mathematics and Economics, Elsevier, vol. 101(PB), pages 485-497.
    21. Frees, Edward W. & Meyers, Glenn & Cummings, A. David, 2011. "Summarizing Insurance Scores Using a Gini Index," Journal of the American Statistical Association, American Statistical Association, vol. 106(495), pages 1085-1098.
    22. repec:cup:astinb:v:5:y:1968:i:01:p:41-48_00 is not listed on IDEAS
    23. Marc-Oliver Pohle, 2020. "The Murphy Decomposition and the Calibration-Resolution Principle: A New Perspective on Forecast Evaluation," Papers 2005.01835, arXiv.org.
    24. Denuit, Michel & Trufin, Julien, 2024. "Convex and Lorenz orders under balance correction in nonlife insurance pricing: Review and new developments," LIDAM Reprints ISBA 2024020, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    25. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    26. Denuit, Michel & Sznajder, Dominik & Trufin, Julien, 2019. "Model selection based on Lorenz and concentration curves, Gini indices and convex order," Insurance: Mathematics and Economics, Elsevier, vol. 89(C), pages 128-139.
    27. Patton, Andrew J., 2011. "Volatility forecast comparison using imperfect volatility proxies," Journal of Econometrics, Elsevier, vol. 160(1), pages 246-256, January.
    28. repec:cup:astinb:v:1:y:1960:i:04:p:192-217_00 is not listed on IDEAS
    29. Denuit, Michel & Sznajder, Dominik & Trufin, Julien, 2019. "Model selection based on Lorenz and concentration curves, Gini indices and convex order," LIDAM Discussion Papers ISBA 2019006, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    30. Denuit, Michel & Huyghe, Julie & Trufin, Julien & Verdebout, Thomas, 2024. "Testing for auto-calibration with Lorenz and Concentration curves," LIDAM Reprints ISBA 2024015, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    31. Mathias Lindholm & Mario V. Wüthrich, 2026. "The balance property in insurance pricing," Scandinavian Actuarial Journal, Taylor & Francis Journals, vol. 2026(5), pages 506-543, May.
    32. Fabian Krüger & Johanna F. Ziegel, 2021. "Generic Conditions for Forecast Dominance," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(4), pages 972-983, October.
    33. Mathias Lindholm & Ronald Richman & Andreas Tsanakas & Mario V. Wuthrich, 2022. "A Discussion of Discrimination and Fairness in Insurance Pricing," Papers 2209.00858, arXiv.org.
    34. Łukasz Delong & Mario V. Wüthrich, 2025. "Isotonic Regression for Variance Estimation and Its Role in Mean Estimation and Model Validation," North American Actuarial Journal, Taylor & Francis Journals, vol. 29(3), pages 563-591, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Denuit, Michel & Trufin, Julien, 2022. "Autocalibration by balance correction in nonlife insurance pricing," LIDAM Discussion Papers ISBA 2022041, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    2. Denuit, Michel & Huyghe, Julie & Trufin, Julien & Verdebout, Thomas, 2024. "Testing for auto-calibration with Lorenz and Concentration curves," Insurance: Mathematics and Economics, Elsevier, vol. 117(C), pages 130-139.
    3. Denuit, Michel & Trufin, Julien, 2024. "Convex and Lorenz orders under balance correction in nonlife insurance pricing: Review and new developments," Insurance: Mathematics and Economics, Elsevier, vol. 118(C), pages 123-128.
    4. Mario V. Wuthrich & Johanna Ziegel, 2023. "Isotonic Recalibration under a Low Signal-to-Noise Ratio," Papers 2301.02692, arXiv.org.
    5. Denuit, Michel & Trufin, Julien & Verdebout, Thomas, 2021. "Testing for more positive expectation dependence with application to model comparison," LIDAM Discussion Papers ISBA 2021021, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    6. Mario V. Wuthrich, 2026. "The Balance Property: The Constrained Case, with a View on Risk Sharing," Papers 2606.07276, arXiv.org.
    7. Fissler, Tobias & Merz, Michael & Wüthrich, Mario V., 2023. "Deep quantile and deep composite triplet regression," Insurance: Mathematics and Economics, Elsevier, vol. 109(C), pages 94-112.
    8. Denuit, Michel & Trufin, Julien, 2022. "Model selection with Pearson’s correlation, concentration and Lorenz curves under autocalibration," LIDAM Discussion Papers ISBA 2022033, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    9. Tobias Fissler & Silvana M. Pesenti, 2022. "Sensitivity Measures Based on Scoring Functions," Papers 2203.00460, arXiv.org, revised Jul 2022.
    10. Denuit, Michel & Trufin, Julien, 2022. "Tweedie dominance for autocalibrated predictors and Laplace transform order," LIDAM Discussion Papers ISBA 2022040, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    11. Jamotton, Charlotte & Hainaut, Donatien, 2025. "A multivariate energy-based fairness adjuster for premiums," LIDAM Discussion Papers ISBA 2025009, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    12. Denuit, Michel & Charpentier, Arthur & Trufin, Julien, 2021. "Autocalibration and Tweedie-dominance for insurance pricing with machine learning," LIDAM Discussion Papers ISBA 2021013, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    13. Arthur Charpentier, 2022. "Quantifying fairness and discrimination in predictive models," Papers 2212.09868, arXiv.org.
    14. Michel Denuit & Arthur Charpentier & Julien Trufin, 2021. "Autocalibration and Tweedie-dominance for Insurance Pricing with Machine Learning," Papers 2103.03635, arXiv.org, revised Jul 2021.
    15. Fissler, Tobias & Pesenti, Silvana M., 2023. "Sensitivity measures based on scoring functions," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1408-1423.
    16. Benjamin Avanzi & Matthew Lambrianidis & Greg Taylor & Bernard Wong, 2025. "On the use of case estimate and transactional payment data in neural networks for individual loss reserving," Papers 2601.05274, arXiv.org.
    17. Hainaut, Donatien, 2025. "In-processing of actuarial and equity fairness constraints for Neural networks," LIDAM Discussion Papers ISBA 2025011, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    18. Yaojun Zhang & Lanpeng Ji & Georgios Aivaliotis & Charles Taylor, 2023. "Bayesian CART models for insurance claims frequency," Papers 2303.01923, arXiv.org, revised Dec 2023.
    19. Chen, Yi-Ting & Liu, Chu-An & Su, Jiun-Hua, 2025. "Bregman model averaging for forecast combination," Journal of Econometrics, Elsevier, vol. 251(C).
    20. Bright Kwaku Manu & Beckett Sterner & Petar Jevtic, 2026. "A Longitudinal Attribute-Conditioned Neural Network for Modeling Health-State Transition Probabilities in Temporally Irregular Data: The LANTERN Framework," Papers 2606.13880, arXiv.org.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2609.26445. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.