IDEAS home Printed from https://ideas.repec.org/p/zbw/esprep/339750.html

Comprehensive Review of Various Verification and Validation Techniques for Business Simulation Models

Author

Listed:
  • Gotherwal, Deepesh
  • Ranjan, Pritam
  • Lekivetz, Ryan

Abstract

Verification and validation (V&V) are integral parts of any simulation study. Validation assesses how accurately conceptual models represent the real system, while verification ensures correct implementation in software. V&V plays a critical role in business and manufacturing, where simulation models imitate complex real-world systems. However, comprehensive statistically grounded literature reviews on V&V of simulation models, particularly from a business management and manufacturing domain standpoint, are scarce. This study addresses that gap by performing topic modeling to identify prominent research themes, then reviewing all important research articles to outline the evolution of quantitative methodologies and algorithms on V&V. We also highlight various research gaps and potential directions for future work. For this study, we reviewed the abstracts of more than 6,000 articles indexed in Scopus and Web of Science, along with a comprehensive analysis of 300 research articles.

Suggested Citation

  • Gotherwal, Deepesh & Ranjan, Pritam & Lekivetz, Ryan, 2026. "Comprehensive Review of Various Verification and Validation Techniques for Business Simulation Models," EconStor Preprints 339750, ZBW - Leibniz Information Centre for Economics.
  • Handle: RePEc:zbw:esprep:339750
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/339750/1/Various-Verification-and-Validation-Techniques.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jack P. C. Kleijnen & Russell C. H. Cheng & Bert Bettonvil, 2001. "Validation of Trace-Driven Simulation Models: Bootstrap Tests," Management Science, INFORMS, vol. 47(11), pages 1533-1538, November.
    2. Kleijnen, Jack P.C. & Deflandre, David, 2006. "Validation of regression metamodels in simulation: Bootstrap approach," European Journal of Operational Research, Elsevier, vol. 170(1), pages 120-131, April.
    3. Aria, Massimo & Cuccurullo, Corrado, 2017. "bibliometrix: An R-tool for comprehensive science mapping analysis," Journal of Informetrics, Elsevier, vol. 11(4), pages 959-975.
    4. Farheen Naz & Rohit Agrawal & Anil Kumar & Angappa Gunasekaran & Abhijit Majumdar & Sunil Luthra, 2022. "Reviewing the applications of artificial intelligence in sustainable supply chains: Exploring research propositions for future directions," Business Strategy and the Environment, Wiley Blackwell, vol. 31(5), pages 2400-2423, July.
    5. Jack P. C. Kleijnen & Wim C. M. van Beers, 2022. "Statistical Tests for Cross-Validation of Kriging Models," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 607-621, January.
    6. Jack P. C. Kleijnen & Bert Bettonvil & Willem Van Groenendaal, 1998. "Validation of Trace-Driven Simulation Models: A Novel Regression Test," Management Science, INFORMS, vol. 44(6), pages 812-819, June.
    7. Barlas, Yaman, 1989. "Multiple tests for validation of system dynamics type of simulation models," European Journal of Operational Research, Elsevier, vol. 42(1), pages 59-87, September.
    8. Pau Fonseca i Casas, 2023. "A Continuous Process for Validation, Verification, and Accreditation of Simulation Models," Mathematics, MDPI, vol. 11(4), pages 1-25, February.
    9. Brailsford, Sally C. & Eldabi, Tillal & Kunc, Martin & Mustafee, Navonil & Osorio, Andres F., 2019. "Hybrid simulation modelling in operational research: A state-of-the-art review," European Journal of Operational Research, Elsevier, vol. 278(3), pages 721-737.
    10. Kleijnen, Jack P. C., 1995. "Verification and validation of simulation models," European Journal of Operational Research, Elsevier, vol. 82(1), pages 145-162, April.
    11. Mustak, Mekhail & Salminen, Joni & Plé, Loïc & Wirtz, Jochen, 2021. "Artificial intelligence in marketing: Topic modeling, scientometric analysis, and research agenda," Journal of Business Research, Elsevier, vol. 124(C), pages 389-404.
    12. Mert Edali, 2022. "Pattern‐oriented analysis of system dynamics models via random forests," System Dynamics Review, System Dynamics Society, vol. 38(2), pages 135-166, April.
    13. Kleijnen, Jack P. C., 1995. "Statistical validation of simulation models," European Journal of Operational Research, Elsevier, vol. 87(1), pages 21-34, November.
    14. Harper, Alison & Mustafee, Navonil & Yearworth, Mike, 2021. "Facets of trust in simulation studies," European Journal of Operational Research, Elsevier, vol. 289(1), pages 197-213.
    15. Dery, Richard & Landry, Maurice & Banville, Claude, 1993. "Revisiting the issue of model validation in OR: An epistemological view," European Journal of Operational Research, Elsevier, vol. 66(2), pages 168-183, April.
    16. Yu Han & Woon Kian Chong & Dong Li, 2020. "A systematic literature review of the capabilities and performance metrics of supply chain resilience," International Journal of Production Research, Taylor & Francis Journals, vol. 58(15), pages 4541-4566, July.
    17. Robinson, Stewart, 2002. "General concepts of quality for discrete-event simulation," European Journal of Operational Research, Elsevier, vol. 138(1), pages 103-117, April.
    18. Thomas H. Naylor & J. M. Finger, 1967. "Verification of Computer Simulation Models," Management Science, INFORMS, vol. 14(2), pages 92-101, October.
    19. Yongxin Liao & Fernando Deschamps & Eduardo de Freitas Rocha Loures & Luiz Felipe Pierin Ramos, 2017. "Past, present and future of Industry 4.0 - a systematic literature review and research agenda proposal," International Journal of Production Research, Taylor & Francis Journals, vol. 55(12), pages 3609-3629, June.
    20. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    21. Kleijnen, Jack P. C., 1983. "Cross-validation using the t statistic," European Journal of Operational Research, Elsevier, vol. 13(2), pages 133-141, June.
    22. Onggo, Bhakti Stephan & Karatas, Mumtaz, 2016. "Test-driven simulation modelling: A case study using agent-based maritime search-operation simulation," European Journal of Operational Research, Elsevier, vol. 254(2), pages 517-531.
    23. Troost, Christian & Huber, Robert & Bell, Andrew R. & van Delden, Hedwig & Filatova, Tatiana & Le, Quang Bao & Lippe, Melvin & Niamir, Leila & Polhill, J. Gareth & Sun, Zhanli & Berger, Thomas, 2023. "How to keep it adequate: A protocol for ensuring validity in agent-based simulation," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 159, pages 1-21.
    24. J Pousi & J Poropudas & K Virtanen, 2013. "Simulation metamodelling with Bayesian networks," Journal of Simulation, Taylor & Francis Journals, vol. 7(4), pages 297-311, November.
    25. Ö Gürcan & O Dikenelli & C Bernon, 2013. "A generic testing framework for agent-based simulation models," Journal of Simulation, Taylor & Francis Journals, vol. 7(3), pages 183-201, August.
    26. R G Sargent, 2015. "An interval statistical procedure for use in validation of simulation models," Journal of Simulation, Taylor & Francis Journals, vol. 9(3), pages 232-237, August.
    27. Kamil Erkan Kabak & Johannes Hinckeldeyn & Rob Dekkers, 2024. "A systematic literature review into simulation for building operations management theory: reaching beyond positivism?," Journal of Simulation, Taylor & Francis Journals, vol. 18(5), pages 687-715, September.
    28. Mustafa Hekimoğlu & Yaman Barlas & Luis Luna-Reyes, 2016. "Sensitivity analysis for models with multiple behavior modes: a method based on behavior pattern measures," System Dynamics Review, System Dynamics Society, vol. 32(3-4), pages 332-362, July.
    29. R G Sargent, 2013. "Verification and validation of simulation models," Journal of Simulation, Taylor & Francis Journals, vol. 7(1), pages 12-24, February.
    30. Tako, Antuela A. & Robinson, Stewart, 2010. "Model development in discrete-event simulation and system dynamics: An empirical study of expert modellers," European Journal of Operational Research, Elsevier, vol. 207(2), pages 784-794, December.
    31. Oral, Muhittin & Kettani, Ossama, 1993. "The facets of the modeling and validation process in operations research," European Journal of Operational Research, Elsevier, vol. 66(2), pages 216-234, April.
    32. W C M van Beers & J P C Kleijnen, 2003. "Kriging for interpolation in random simulation," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 54(3), pages 255-262, March.
    33. Sheldon H. Jacobson & Enver Yücesan, 1999. "On the Complexity of Verifying Structural Properties of Discrete Event Simulation Models," Operations Research, INFORMS, vol. 47(3), pages 476-481, June.
    34. Kleijnen, Jack P. C. & Sargent, Robert G., 2000. "A methodology for fitting and validating metamodels in simulation," European Journal of Operational Research, Elsevier, vol. 120(1), pages 14-29, January.
    35. Mekhail Mustak & Joni Salminen & Loïc Plé & Jochen Wirtz, 2021. "Artificial intelligence in marketing: Topic modeling, scientometric analysis, and research agenda," Post-Print hal-03269994, HAL.
    36. Landry, Maurice & Oral, Muhittin, 1993. "In search of a valid view of model validation for operations research," European Journal of Operational Research, Elsevier, vol. 66(2), pages 161-167, April.
    37. Nees Jan Eck & Ludo Waltman, 2010. "Software survey: VOSviewer, a computer program for bibliometric mapping," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(2), pages 523-538, August.
    38. Dimitris Mourtzis, 2020. "Simulation in the design and operation of manufacturing systems: state of the art and new trends," International Journal of Production Research, Taylor & Francis Journals, vol. 58(7), pages 1927-1949, April.
    39. Strang, Kenneth David, 2012. "Importance of verifying queue model assumptions before planning with simulation software," European Journal of Operational Research, Elsevier, vol. 218(2), pages 493-504.
    40. Vishwas Dohale & Angappa Gunasekaran & Milind Madhukarro Akarte & Priyanka Verma, 2022. "52 Years of manufacturing strategy: an evolutionary review of literature (1969–2021)," International Journal of Production Research, Taylor & Francis Journals, vol. 60(2), pages 569-594, January.
    41. Kleijnen, J.P.C., 1995. "Statistical validation of simulation models : A case study," Other publications TiSEM 4da192cf-c3f4-40a4-aea4-5, Tilburg University, School of Economics and Management.
    42. Landry, Maurice & Malouin, Jean-Louis & Oral, Muhittin, 1983. "Model validation in operations research," European Journal of Operational Research, Elsevier, vol. 14(3), pages 207-220, November.
    43. Foivos Psarommatis & Gokan May, 2023. "A literature review and design methodology for digital twins in the era of zero defect manufacturing," International Journal of Production Research, Taylor & Francis Journals, vol. 61(16), pages 5723-5743, August.
    44. Jahangirian, Mohsen & Eldabi, Tillal & Naseer, Aisha & Stergioulas, Lampros K. & Young, Terry, 2010. "Simulation in manufacturing and business: A review," European Journal of Operational Research, Elsevier, vol. 203(1), pages 1-13, May.
    45. Kleijnen, Jack P.C., 2017. "Regression and Kriging metamodels with their experimental designs in simulation: A review," European Journal of Operational Research, Elsevier, vol. 256(1), pages 1-16.
    46. Stewart Robinson, 2020. "Conceptual modelling for simulation: Progress and grand challenges," Journal of Simulation, Taylor & Francis Journals, vol. 14(1), pages 1-20, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Janová, Jitka & Hampel, David & Nerudová, Danuše, 2019. "Design and validation of a tax sustainability index," European Journal of Operational Research, Elsevier, vol. 278(3), pages 916-926.
    2. Kleijnen, J.P.C., 2006. "Regression Models and Experimental Designs : A Tutorial for Simulation Analaysts," Discussion Paper 2006-10, Tilburg University, Center for Economic Research.
    3. Arnold Reisman & Muhittin Oral, 2005. "Soft Systems Methodology: A Context Within a 50-Year Retrospective of OR/MS," Interfaces, INFORMS, vol. 35(2), pages 164-178, April.
    4. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    5. Kleijnen, Jack P. C. & Sargent, Robert G., 2000. "A methodology for fitting and validating metamodels in simulation," European Journal of Operational Research, Elsevier, vol. 120(1), pages 14-29, January.
    6. Zhao Zhi-hong & Wei Zi-yi & Yao Shan-ji, 2025. "How Entrepreneurship University Enhances Digital Opportunities? Evidences from Bibliometric and Topic Modeling," SAGE Open, , vol. 15(3), pages 21582440251, August.
    7. Strang, Kenneth David, 2012. "Importance of verifying queue model assumptions before planning with simulation software," European Journal of Operational Research, Elsevier, vol. 218(2), pages 493-504.
    8. Halachmi, I. & Dzidic, A. & Metz, J. H. M. & Speelman, L. & Dijkhuizen, A. A. & Kleijnen, J. P. C., 2001. "Validation of simulation model for robotic milking barn design," European Journal of Operational Research, Elsevier, vol. 134(3), pages 677-688, November.
    9. Robinson, Stewart, 2002. "General concepts of quality for discrete-event simulation," European Journal of Operational Research, Elsevier, vol. 138(1), pages 103-117, April.
    10. Ilan Halachmi & Yitzhak Simon & Noam Mozes, 2014. "Simulation of the shift from marine netcages to inland recirculating aquaculture systems," Annals of Operations Research, Springer, vol. 219(1), pages 85-99, August.
    11. Donthu, Naveen & Kumar, Satish & Mukherjee, Debmalya & Pandey, Nitesh & Lim, Weng Marc, 2021. "How to conduct a bibliometric analysis: An overview and guidelines," Journal of Business Research, Elsevier, vol. 133(C), pages 285-296.
    12. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    13. Mariani, Marcello M. & Hashemi, Novin & Wirtz, Jochen, 2023. "Artificial intelligence empowered conversational agents: A systematic literature review and research agenda," Journal of Business Research, Elsevier, vol. 161(C).
    14. Andrew J Collins & Farinaz Sabz Ali Pour & Craig A Jordan, 2023. "Past challenges and the future of discrete event simulation," The Journal of Defense Modeling and Simulation, , vol. 20(3), pages 351-369, July.
    15. Manuel Muth & Michael Lingenfelder & Gerd Nufer, 2025. "The application of machine learning for demand prediction under macroeconomic volatility: a systematic literature review," Management Review Quarterly, Springer, vol. 75(3), pages 2759-2802, September.
    16. Giacomo Zatini, 2025. "Conditions of use and impacts of artificial intelligence in marketing practices: a mixed-method literature review," Italian Journal of Marketing, Springer, vol. 2025(3), pages 293-329, September.
    17. Levent Yilmaz, 2006. "Validation and verification of social processes within agent-based computational organization models," Computational and Mathematical Organization Theory, Springer, vol. 12(4), pages 283-312, December.
    18. Kleijnen, J.P.C. & Sanchez, S.M. & Lucas, T.W. & Cioppa, T.M., 2003. "A User's Guide to the Brave New World of Designing Simulation Experiments," Discussion Paper 2003-1, Tilburg University, Center for Economic Research.
    19. Kleijnen, J. P. C., 2001. "Ethical issues in modeling: Some reflections," European Journal of Operational Research, Elsevier, vol. 130(1), pages 223-230, April.
    20. Fraedrich, D. & Goldberg, A., 2000. "A methodological framework for the validation of predictive simulations," European Journal of Operational Research, Elsevier, vol. 124(1), pages 55-62, July.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:esprep:339750. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.