IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v347y2025i2d10.1007_s10479-024-05943-4.html
   My bibliography  Save this article

Quantifying and explaining machine learning uncertainty in predictive process monitoring: an operations research perspective

Author

Listed:
  • Nijat Mehdiyev

    (German Research Center for Artificial Intelligence (DFKI)
    Saarland University)

  • Maxim Majlatow

    (German Research Center for Artificial Intelligence (DFKI)
    Saarland University)

  • Peter Fettke

    (German Research Center for Artificial Intelligence (DFKI)
    Saarland University)

Abstract

In the rapidly evolving landscape of manufacturing, the ability to make accurate predictions is crucial for optimizing processes. This study introduces a novel framework that combines predictive uncertainty with explanatory mechanisms to enhance decision-making in complex systems. The approach leverages Quantile Regression Forests for reliable predictive process monitoring and incorporates Shapley Additive Explanations (SHAP) to identify the drivers of predictive uncertainty. This dual-faceted strategy serves as a valuable tool for domain experts engaged in process planning activities. Supported by a real-world case study involving a medium-sized German manufacturing firm, the article validates the model’s effectiveness through rigorous evaluations, including sensitivity analyses and tests for statistical significance. By seamlessly integrating uncertainty quantification with explainable artificial intelligence, this research makes a novel contribution to the evolving discourse on intelligent decision-making in complex systems.

Suggested Citation

  • Nijat Mehdiyev & Maxim Majlatow & Peter Fettke, 2025. "Quantifying and explaining machine learning uncertainty in predictive process monitoring: an operations research perspective," Annals of Operations Research, Springer, vol. 347(2), pages 991-1030, April.
  • Handle: RePEc:spr:annopr:v:347:y:2025:i:2:d:10.1007_s10479-024-05943-4
    DOI: 10.1007/s10479-024-05943-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-024-05943-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10479-024-05943-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tavares Thomé, Antônio Márcio & Scavarda, Luiz Felipe & Fernandez, Nicole Suclla & Scavarda, Annibal José, 2012. "Sales and operations planning: A research synthesis," International Journal of Production Economics, Elsevier, vol. 138(1), pages 1-13.
    2. Willem van Jaarsveld & Alan Scheller-Wolf, 2015. "Optimization of Industrial-Scale Assemble-to-Order Systems," INFORMS Journal on Computing, INFORMS, vol. 27(3), pages 544-560, August.
    3. Zhengliang Xue & Zizhuo Wang & Markus Ettl, 2016. "Pricing Personalized Bundles: A New Approach and An Empirical Study," Manufacturing & Service Operations Management, INFORMS, vol. 18(1), pages 51-68, February.
    4. Dimitris Bertsimas & Nathan Kallus, 2020. "From Predictive to Prescriptive Analytics," Management Science, INFORMS, vol. 66(3), pages 1025-1044, March.
    5. Juan Pablo Usuga Cadavid & Samir Lamouri & Bernard Grabot & Robert Pellerin & Arnaud Fortin, 2020. "Machine learning applied in production planning and control: a state-of-the-art in the era of industry 4.0," Journal of Intelligent Manufacturing, Springer, vol. 31(6), pages 1531-1558, August.
    6. Lepenioti, Katerina & Bousdekis, Alexandros & Apostolou, Dimitris & Mentzas, Gregoris, 2020. "Prescriptive analytics: Literature review and research challenges," International Journal of Information Management, Elsevier, vol. 50(C), pages 57-70.
    7. Bengio, Yoshua & Lodi, Andrea & Prouvost, Antoine, 2021. "Machine learning for combinatorial optimization: A methodological tour d’horizon," European Journal of Operational Research, Elsevier, vol. 290(2), pages 405-421.
    8. Sai Ho Chung & Hoi Lam Ma & Hing Kai Chan, 2017. "Cascading Delay Risk of Airline Workforce Deployments with Crew Pairing and Schedule Optimization," Risk Analysis, John Wiley & Sons, vol. 37(8), pages 1443-1458, August.
    9. Seokjun Youn & H. Neil Geismar & Michael Pinedo, 2022. "Planning and scheduling in healthcare for better care coordination: Current understanding, trending topics, and future opportunities," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4407-4423, December.
    10. Mitrentsis, Georgios & Lens, Hendrik, 2022. "An interpretable probabilistic model for short-term solar power forecasting using natural gradient boosting," Applied Energy, Elsevier, vol. 309(C).
    11. Ramesh Kumar & L. Ganapathy & Ravindra Gokhale & Manoj Kumar Tiwari, 2020. "Quantitative approaches for the integration of production and distribution planning in the supply chain: a systematic literature review," International Journal of Production Research, Taylor & Francis Journals, vol. 58(11), pages 3527-3553, June.
    12. Mula, J. & Poler, R. & Garcia-Sabater, J.P. & Lario, F.C., 2006. "Models for production planning under uncertainty: A review," International Journal of Production Economics, Elsevier, vol. 103(1), pages 271-285, September.
    13. Herbert Simon, 1997. "The future of information systems," Annals of Operations Research, Springer, vol. 71(0), pages 3-14, January.
    14. Velibor V. Mišić & Georgia Perakis, 2020. "Data Analytics in Operations Management: A Review," Manufacturing & Service Operations Management, INFORMS, vol. 22(1), pages 158-169, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Muzaffer Buyruk & Ertan Güner, 2022. "Personalization in airline revenue management: an overview and future outlook," Journal of Revenue and Pricing Management, Palgrave Macmillan, vol. 21(2), pages 129-139, April.
    2. Filom, Siyavash & Amiri, Amir M. & Razavi, Saiedeh, 2022. "Applications of machine learning methods in port operations – A systematic literature review," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 161(C).
    3. Schmidt, Felix G. & Pibernik, Richard, 2025. "Data-driven inventory control for large product portfolios: A practical application of prescriptive analytics," European Journal of Operational Research, Elsevier, vol. 322(1), pages 254-269.
    4. Sel, Burakhan & Minner, Stefan, 2022. "A hedging policy for seaborne forward freight markets based on probabilistic forecasts," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 166(C).
    5. Akhtar, Pervaiz & Ghouri, Arsalan Mujahid & Ashraf, Aniqa & Lim, Jia Jia & Khan, Naveed R & Ma, Shuang, 2024. "Smart product platforming powered by AI and generative AI: Personalization for the circular economy," International Journal of Production Economics, Elsevier, vol. 273(C).
    6. Jung, Seung Hwan & Yang, Yunsi, 2023. "On the value of operational flexibility in the trailer shipment and assignment problem: Data-driven approaches and reinforcement learning," International Journal of Production Economics, Elsevier, vol. 264(C).
    7. Wang, Shuaian & Yan, Ran, 2023. "Fundamental challenge and solution methods in prescriptive analytics for freight transportation," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 169(C).
    8. Laurent Lim, Lâm & Alpan, Gülgün & Penz, Bernard, 2014. "Reconciling sales and operations management with distant suppliers in the automotive industry: A simulation approach," International Journal of Production Economics, Elsevier, vol. 151(C), pages 20-36.
    9. Qi Feng & J. George Shanthikumar, 2023. "The framework of parametric and nonparametric operational data analytics," Production and Operations Management, Production and Operations Management Society, vol. 32(9), pages 2685-2703, September.
    10. Adam N. Elmachtoub & Paul Grigas, 2022. "Smart “Predict, then Optimize”," Management Science, INFORMS, vol. 68(1), pages 9-26, January.
    11. Shaochong Lin & Youhua (Frank) Chen & Yanzhi Li & Zuo‐Jun Max Shen, 2022. "Data‐Driven Newsvendor Problems Regularized by a Profit Risk Constraint," Production and Operations Management, Production and Operations Management Society, vol. 31(4), pages 1630-1644, April.
    12. Long He & Sheng Liu & Zuo‐Jun Max Shen, 2022. "Smart urban transport and logistics: A business analytics perspective," Production and Operations Management, Production and Operations Management Society, vol. 31(10), pages 3771-3787, October.
    13. Erkip, Nesim Kohen, 2023. "Can accessing much data reshape the theory? Inventory theory under the challenge of data-driven systems," European Journal of Operational Research, Elsevier, vol. 308(3), pages 949-959.
    14. Yinchu Zhu & Ilya O. Ryzhov, 2022. "Optimal data-driven hiring with equity for underrepresented groups," Papers 2206.09300, arXiv.org.
    15. Fajemisin, Adejuyigbe O. & Maragno, Donato & den Hertog, Dick, 2024. "Optimization with constraint learning: A framework and survey," European Journal of Operational Research, Elsevier, vol. 314(1), pages 1-14.
    16. Pereira, Daniel Filipe & Oliveira, José Fernando & Carravilla, Maria Antónia, 2023. "Design of a sales plan in a hybrid contractual and non-contractual context in a setting of limited capacity: A robust approach," International Journal of Production Economics, Elsevier, vol. 260(C).
    17. Sadana, Utsav & Chenreddy, Abhilash & Delage, Erick & Forel, Alexandre & Frejinger, Emma & Vidal, Thibaut, 2025. "A survey of contextual optimization methods for decision-making under uncertainty," European Journal of Operational Research, Elsevier, vol. 320(2), pages 271-289.
    18. Hauser, Matthias & Flath, Christoph M. & Thiesse, Frédéric, 2021. "Catch me if you scan: Data-driven prescriptive modeling for smart store environments," European Journal of Operational Research, Elsevier, vol. 294(3), pages 860-873.
    19. Qi Feng & J. George Shanthikumar, 2022. "Developing operations management data analytics," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4544-4557, December.
    20. Jonsson, Patrik & Kjellsdotter Ivert, Linea, 2015. "Improving performance with sophisticated master production scheduling," International Journal of Production Economics, Elsevier, vol. 168(C), pages 118-130.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:347:y:2025:i:2:d:10.1007_s10479-024-05943-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.