IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v195y2009i3p642-652.html
   My bibliography  Save this article

An integrative evaluation framework for intelligent decision support systems

Author

Listed:
  • Phillips-Wren, G.
  • Mora, M.
  • Forgionne, G.A.
  • Gupta, J.N.D.

Abstract

Evaluation of the overall effectiveness of decision support systems (DSS) has been a research topic since the early 1980s. As artificial intelligence methods have been incorporated into systems to create intelligent decision support systems (IDSS), researchers have attempted to quantify the value of the additional capabilities. Despite the useful and relevant insights generated by previous research, existing evaluation methodologies offer only a fragmented and incomplete view of IDSS value and the contribution of its technical infrastructure. This paper proposes an integrative, multiple criteria IDSS evaluation framework through a model that links the decision value of an IDSS to both the outcome from, and process of, decision making and down to specific components of the IDSS. The proposed methodology provides the designer and developer specific guidance on the intelligent tools most useful for a specific user with a particular decision problem. The proposed framework is illustrated by evaluating an actual IDSS that coordinates management of urban infrastructures.

Suggested Citation

  • Phillips-Wren, G. & Mora, M. & Forgionne, G.A. & Gupta, J.N.D., 2009. "An integrative evaluation framework for intelligent decision support systems," European Journal of Operational Research, Elsevier, vol. 195(3), pages 642-652, June.
    • Handle: RePEc:eee:ejores:v:195:y:2009:i:3:p:642-652
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377-2217(07)01077-6
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Herbert A. Simon, 1987. "Two Heads Are Better than One: The Collaboration between AI and OR," Interfaces, INFORMS, vol. 17(4), pages 8-15, August.
    2. Rajiv Kohli & Sarv Devaraj, 2003. "Measuring Information Technology Payoff: A Meta-Analysis of Structural Variables in Firm-Level Empirical Research," Information Systems Research, INFORMS, vol. 14(2), pages 127-145, June.
    3. Rosenthal-Sabroux, Camille & Zarate, Pascale, 1997. "Artificial intelligence tools for decision support systems," European Journal of Operational Research, Elsevier, vol. 103(2), pages 275-276, December.
    4. Yoram Wind & Thomas L. Saaty, 1980. "Marketing Applications of the Analytic Hierarchy Process," Management Science, INFORMS, vol. 26(7), pages 641-658, July.
    5. Dennis M. Buede, 1986. "Structuring Value Attributes," Interfaces, INFORMS, vol. 16(2), pages 52-62, April.
    6. Ramesh Sharda & Steve H. Barr & James C. McDonnell, 1988. "Decision Support System Effectiveness: A Review and an Empirical Test," Management Science, INFORMS, vol. 34(2), pages 139-159, February.
    7. Quintero, Alejandro & Konare, Dougoukolo & Pierre, Samuel, 2005. "Prototyping an intelligent decision support system for improving urban infrastructures management," European Journal of Operational Research, Elsevier, vol. 162(3), pages 654-672, May.
    8. Sarv Devaraj & Rajiv Kohli, 2003. "Performance Impacts of Information Technology: Is Actual Usage the Missing Link?," Management Science, INFORMS, vol. 49(3), pages 273-289, March.
    9. Phillips-Wren, Gloria E. & Hahn, Eugene D. & Forgionne, Guisseppi A., 2004. "A multiple-criteria framework for evaluation of decision support systems," Omega, Elsevier, vol. 32(4), pages 323-332, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shivam Gupta & Sachin Modgil & Samadrita Bhattacharyya & Indranil Bose, 2022. "Artificial intelligence for decision support systems in the field of operations research: review and future scope of research," Annals of Operations Research, Springer, vol. 308(1), pages 215-274, January.
    2. Afshan Naseem & Yasir Ahmad, 2020. "Critical Success Factors for Neutralization of Airborne Threats," SAGE Open, , vol. 10(3), pages 21582440209, September.
    3. Zhang, Ruijun & Lu, Jie & Zhang, Guangquan, 2011. "A knowledge-based multi-role decision support system for ore blending cost optimization of blast furnaces," European Journal of Operational Research, Elsevier, vol. 215(1), pages 194-203, November.
    4. Christoph Keding, 2021. "Understanding the interplay of artificial intelligence and strategic management: four decades of research in review," Management Review Quarterly, Springer, vol. 71(1), pages 91-134, February.
    5. Afshan Naseem & Shoab Ahmed Khan & Asad Waqar Malik, 2017. "A real-time man-in-loop threat evaluation and resource assignment in defense," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(6), pages 725-738, June.
    6. Manuel Mora & Gloria Phillips-Wren & Fen Wang & Ovsei Gelman, 2017. "An Exploratory-Comparative Study of Implementation Success Factors for MSS/DMSS and MIS," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 16(06), pages 1671-1705, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eymann, Torsten (Ed.), 2009. "Tagungsband zum Doctoral Consortium der WI 2009 [WI2009 Doctoral Consortium Proceedings]," Bayreuth Reports on Information Systems Management 40, University of Bayreuth, Chair of Information Systems Management.
    2. Sinan Aral & Peter Weill, 2007. "IT Assets, Organizational Capabilities, and Firm Performance: How Resource Allocations and Organizational Differences Explain Performance Variation," Organization Science, INFORMS, vol. 18(5), pages 763-780, October.
    3. Polák, Petr, 2017. "The productivity paradox: A meta-analysis," Information Economics and Policy, Elsevier, vol. 38(C), pages 38-54.
    4. Philipp Köllinger, 2005. "Why IT Matters: An Empirical Study of E-Business Usage, Innovation, and Firm Performance," Discussion Papers of DIW Berlin 495, DIW Berlin, German Institute for Economic Research.
    5. Emanuele Gabriel Margherita & Alessio Maria Braccini, 2023. "Industry 4.0 Technologies in Flexible Manufacturing for Sustainable Organizational Value: Reflections from a Multiple Case Study of Italian Manufacturers," Information Systems Frontiers, Springer, vol. 25(3), pages 995-1016, June.
    6. Rajiv D. Banker & Robert J. Kauffman, 2004. "50th Anniversary Article: The Evolution of Research on Information Systems: A Fiftieth-Year Survey of the Literature in Management Science," Management Science, INFORMS, vol. 50(3), pages 281-298, March.
    7. Razali Bin Mat, 2012. "Human Resource Management (HRM) and Information Technology (IT): Some Empirical Evidences in the Context of Saudi Arabia," Journal of Education and Vocational Research, AMH International, vol. 3(1), pages 28-34.
    8. Stefan Schweikl & Robert Obermaier, 2020. "Lessons from three decades of IT productivity research: towards a better understanding of IT-induced productivity effects," Management Review Quarterly, Springer, vol. 70(4), pages 461-507, November.
    9. Koutroumpis, Pantelis & Leiponen, Aija & Thomas, Llewellyn D W, 2017. "The Young, the Old and the Innovative: The Impact of R&D on Firm Performance in ICT versus Other Sectors," ETLA Working Papers 51, The Research Institute of the Finnish Economy.
    10. Samppa Suoniemi & Lars Meyer-Waarden & Andreas Munzel & Alex Ricardo Zablah & Detmar Straub, 2020. "Big data and firm performance: The roles of market-directed capabilities and business strategy," Post-Print hal-02957479, HAL.
    11. Michael Ahearne & Eli Jones & Adam Rapp & John Mathieu, 2008. "High Touch Through High Tech: The Impact of Salesperson Technology Usage on Sales Performance via Mediating Mechanisms," Management Science, INFORMS, vol. 54(4), pages 671-685, April.
    12. Yajiong Xue & Huigang Liang & Liansheng Wu, 2011. "Punishment, Justice, and Compliance in Mandatory IT Settings," Information Systems Research, INFORMS, vol. 22(2), pages 400-414, June.
    13. Indranil Bardhan & Viswanathan Krishnan & Shu Lin, 2013. "Research Note ---Business Value of Information Technology: Testing the Interaction Effect of IT and R&D on Tobin's Q," Information Systems Research, INFORMS, vol. 24(4), pages 1147-1161, December.
    14. Karim, Mohammed Shamsul & Nahar, Sharmin & Demirbag, Mehmet, 2022. "Resource-Based Perspective on ICT Use and Firm Performance: A Meta-analysis Investigating the Moderating Role of Cross-Country ICT Development Status," Technological Forecasting and Social Change, Elsevier, vol. 179(C).
    15. Sarv Devaraj & Rajiv Kohli, 2003. "Performance Impacts of Information Technology: Is Actual Usage the Missing Link?," Management Science, INFORMS, vol. 49(3), pages 273-289, March.
    16. Paul A. Pavlou & Omar A. El Sawy, 2006. "From IT Leveraging Competence to Competitive Advantage in Turbulent Environments: The Case of New Product Development," Information Systems Research, INFORMS, vol. 17(3), pages 198-227, September.
    17. Juul Andersen, Torben, 2005. "The performance effect of computer-mediated communication and decentralized strategic decision making," Journal of Business Research, Elsevier, vol. 58(8), pages 1059-1067, August.
    18. Emanuele Gabriel Margherita & Alessio Maria Braccini, 2021. "Industry 4.0 Technologies in Flexible Manufacturing for Sustainable Organizational Value: Reflections from a Multiple Case Study of Italian Manufacturers," Post-Print hal-03442440, HAL.
    19. K. D. Joshi & Lei Chi & Avimanyu Datta & Shu Han, 2010. "Changing the Competitive Landscape: Continuous Innovation Through IT-Enabled Knowledge Capabilities," Information Systems Research, INFORMS, vol. 21(3), pages 472-495, September.
    20. Banai, Reza, 2010. "Evaluation of land use-transportation systems with the Analytic Network Process," The Journal of Transport and Land Use, Center for Transportation Studies, University of Minnesota, vol. 3(1), pages 85-112.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:195:y:2009:i:3:p:642-652. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.