IDEAS home Printed from https://ideas.repec.org/a/eee/teinso/v78y2024ics0160791x24002100.html

Exploring collaborative decision-making: A quasi-experimental study of human and Generative AI interaction

Author

Listed:
  • Hao, Xinyue
  • Demir, Emrah
  • Eyers, Daniel

Abstract

This paper explores the effects of integrating Generative Artificial Intelligence (GAI) into decision-making processes within organizations, employing a quasi-experimental pretest-posttest design. The study examines the synergistic interaction between Human Intelligence (HI) and GAI across four group decision-making scenarios within three global organizations renowned for their cutting-edge operational techniques. The research progresses through several phases: identifying research problems, collecting baseline data on decision-making, implementing AI interventions, and evaluating the outcomes post-intervention to identify shifts in performance. The results demonstrate that GAI effectively reduces human cognitive burdens and mitigates heuristic biases by offering data-driven support and predictive analytics, grounded in System 2 reasoning. This is particularly valuable in complex situations characterized by unfamiliarity and information overload, where intuitive, System 1 thinking is less effective. However, the study also uncovers challenges related to GAI integration, such as potential over-reliance on technology, intrinsic biases particularly ‘out-of-the-box’ thinking without contextual creativity. To address these issues, this paper proposes an innovative strategic framework for HI-GAI collaboration that emphasizes transparency, accountability, and inclusiveness.

Suggested Citation

  • Hao, Xinyue & Demir, Emrah & Eyers, Daniel, 2024. "Exploring collaborative decision-making: A quasi-experimental study of human and Generative AI interaction," Technology in Society, Elsevier, vol. 78(C).
    • Handle: RePEc:eee:teinso:v:78:y:2024:i:c:s0160791x24002100
    DOI: 10.1016/j.techsoc.2024.102662
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0160791X24002100
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techsoc.2024.102662?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Andreas Glöckner & Tilmann Betsch, 2008. "Modeling Option and Strategy Choices with Connectionist Networks: Towards an Integrative Model of Automatic and Deliberate Decision Making," Discussion Paper Series of the Max Planck Institute for Behavioral Economics 2008_02, Max Planck Institute for Behavioral Economics.
    2. Coeckelbergh, Mark & Sætra, Henrik Skaug, 2023. "Climate change and the political pathways of AI: The technocracy-democracy dilemma in light of artificial intelligence and human agency," Technology in Society, Elsevier, vol. 75(C).
    3. Jarrahi, Mohammad Hossein, 2018. "Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making," Business Horizons, Elsevier, vol. 61(4), pages 577-586.
    4. Duan, Yanqing & Edwards, John S. & Dwivedi, Yogesh K, 2019. "Artificial intelligence for decision making in the era of Big Data – evolution, challenges and research agenda," International Journal of Information Management, Elsevier, vol. 48(C), pages 63-71.
    5. Sutton, Steve G. & Arnold, Vicky, 2013. "Focus group methods: Using interactive and nominal groups to explore emerging technology-driven phenomena in accounting and information systems," International Journal of Accounting Information Systems, Elsevier, vol. 14(2), pages 81-88.
    6. Ostheimer, Julia & Chowdhury, Soumitra & Iqbal, Sarfraz, 2021. "An alliance of humans and machines for machine learning: Hybrid intelligent systems and their design principles," Technology in Society, Elsevier, vol. 66(C).
    7. Krawinkler, Andreas & Breitenecker, Robert J. & Maresch, Daniela, 2022. "Heuristic decision-making in the green energy context:Bringing together simple rules and data-driven mathematical optimization," Technological Forecasting and Social Change, Elsevier, vol. 180(C).
    8. Monica D. Rosenberg & B. J. Casey & Avram J. Holmes, 2018. "Prediction complements explanation in understanding the developing brain," Nature Communications, Nature, vol. 9(1), pages 1-13, December.
    9. Glöckner, Andreas & Betsch, Tilmann, 2008. "Modeling option and strategy choices with connectionist networks: Towards an integrative model of automatic and deliberate decision making," Judgment and Decision Making, Cambridge University Press, vol. 3(3), pages 215-228, March.
    10. Villa, Sebastián & Castañeda, Jaime Andrés, 2018. "Transshipments in supply chains: A behavioral investigation," European Journal of Operational Research, Elsevier, vol. 269(2), pages 715-729.
    11. Charles Ka Yui Leung & Joe Cho Yiu Ng & Edward Chi Ho TANG, 2019. "What do we know about Housing Supply? The case of Hong Kong," GRU Working Paper Series GRU_2019_012, City University of Hong Kong, Department of Economics and Finance, Global Research Unit.
    12. Samadhiya, Ashutosh & Yadav, Sanjeev & Kumar, Anil & Majumdar, Abhijit & Luthra, Sunil & Garza-Reyes, Jose Arturo & Upadhyay, Arvind, 2023. "The influence of artificial intelligence techniques on disruption management: Does supply chain dynamism matter?," Technology in Society, Elsevier, vol. 75(C).
    13. Haris Krijestorac & Rajiv Garg & Prabhudev Konana, 2021. "Decisions Under the Illusion of Objectivity: Digital Embeddedness and B2B Purchasing," Production and Operations Management, Production and Operations Management Society, vol. 30(7), pages 2232-2251, July.
    14. Huang, Lei & Ladikas, Miltos & Schippl, Jens & He, Guangxi & Hahn, Julia, 2023. "Knowledge mapping of an artificial intelligence application scenario: A bibliometric analysis of the basic research of data-driven autonomous vehicles," Technology in Society, Elsevier, vol. 75(C).
    15. Yuhui Jing & Haoming Wang & Xiaojiao Chen & Chengliang Wang, 2024. "What factors will affect the effectiveness of using ChatGPT to solve programming problems? A quasi-experimental study," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 11(1), pages 1-12, December.
    16. Li, Jing-Ping & Mirza, Nawazish & Rahat, Birjees & Xiong, Deping, 2020. "Machine learning and credit ratings prediction in the age of fourth industrial revolution," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    17. Berman, Alexander & de Fine Licht, Karl & Carlsson, Vanja, 2024. "Trustworthy AI in the public sector: An empirical analysis of a Swedish labor market decision-support system," Technology in Society, Elsevier, vol. 76(C).
    18. Nima Kordzadeh & Maryam Ghasemaghaei, 2022. "Algorithmic bias: review, synthesis, and future research directions," European Journal of Information Systems, Taylor & Francis Journals, vol. 31(3), pages 388-409, May.
    19. Andreas Glöckner & Tilmann Betsch, 2008. "Modelling option and strategy choices with connectionist networks: Towards an integrative model of automatic and deliberate decision making," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 3, pages 215-228, March.
    20. Giordano, Vito & Spada, Irene & Chiarello, Filippo & Fantoni, Gualtiero, 2024. "The impact of ChatGPT on human skills: A quantitative study on twitter data," Technological Forecasting and Social Change, Elsevier, vol. 203(C).
    21. Khogali, Hisham O. & Mekid, Samir, 2023. "The blended future of automation and AI: Examining some long-term societal and ethical impact features," Technology in Society, Elsevier, vol. 73(C).
    22. Sharif, Amir M. & Irani, Zahir, 2006. "Exploring Fuzzy Cognitive Mapping for IS Evaluation," European Journal of Operational Research, Elsevier, vol. 173(3), pages 1175-1187, September.
    23. Hangl, Johannes & Krause, Simon & Behrens, Viktoria Joy, 2023. "Drivers, barriers and social considerations for AI adoption in SCM," Technology in Society, Elsevier, vol. 74(C).
    24. Jackson, Thomas W. & Farzaneh, Pourya, 2012. "Theory-based model of factors affecting information overload," International Journal of Information Management, Elsevier, vol. 32(6), pages 523-532.
    25. Emrah Demir & Katy Huckle & Aris Syntetos & Andrew Lahy & Mike Wilson, 2019. "Vehicle Routing Problem: Past and Future," Springer Books, in: Peter Wells (ed.), Contemporary Operations and Logistics, chapter 0, pages 97-117, Springer.
    26. Meissner, Philip & Wulf, Torsten, 2013. "Cognitive benefits of scenario planning: Its impact on biases and decision quality," Technological Forecasting and Social Change, Elsevier, vol. 80(4), pages 801-814.
    27. Christoph Werner & Anca M. Hanea & Oswaldo Morales-Nápoles, 2018. "Eliciting Multivariate Uncertainty from Experts: Considerations and Approaches Along the Expert Judgement Process," International Series in Operations Research & Management Science, in: Luis C. Dias & Alec Morton & John Quigley (ed.), Elicitation, chapter 0, pages 171-210, Springer.
    28. Bin-Nashwan, Saeed Awadh & Sadallah, Mouad & Bouteraa, Mohamed, 2023. "Use of ChatGPT in academia: Academic integrity hangs in the balance," Technology in Society, Elsevier, vol. 75(C).
    29. Scott, Rodney J & Cavana, Robert Y & Cameron, Donald, 2016. "Recent evidence on the effectiveness of group model building," European Journal of Operational Research, Elsevier, vol. 249(3), pages 908-918.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ayyildiz, Ahu Yazici & Ayyildiz, Tugrul & Koc, Erdogan, 2026. "The use of ChatGPT in service recovery: Compensating customers," Technology in Society, Elsevier, vol. 84(C).
    2. Ji Ma & Albert Casella, 2025. "Integrating Transparent Models, LLMs, and Practitioner-in-the-Loop: A Case of Nonprofit Program Evaluation," Papers 2510.19799, arXiv.org.
    3. George Spais & George Chryssochoidis, 2025. "Trends and future of artificial intelligence (AI), machine learning (ML) algorithms, and data analytics and their applications and implications for digital marketing and digital promotions," Journal of Marketing Analytics, Palgrave Macmillan, vol. 13(2), pages 263-266, June.
    4. Qian, Yiming & Wan, Xiaoang, 2025. "Co-experiencing with AI: Effects on social bonding and empathy in human-AI relationships," Technology in Society, Elsevier, vol. 83(C).
    5. Bai, Shizhen & Zhou, Jiamin & Han, Chunjia & Yang, Mu & Wang, Chen & Gupta, Brij B., 2026. "From information to operations: Exploring the impact of integrated AI-HI on information diffusion and users' willingness in humanitarian aid," Technological Forecasting and Social Change, Elsevier, vol. 224(C).
    6. Ghosh, Soumili & Behera, Rajat Kumar & Bala, Pradip Kumar & Rana, Nripendra P., 2025. "Adverse impacts of metaverse-induced cognitive biases on the immersive shopping experience: A conceptual model developed from a qualitative approach," Technology in Society, Elsevier, vol. 82(C).
    7. Wang, Mingsheng & Yang, Yongzhong & Zhong, Peichi, 2025. "How does AI affect the self-actualization of content creators in dynamic environments? A knowledge management perspective," Technology in Society, Elsevier, vol. 81(C).
    8. Basilio Ruiz de Apodaca, Oihana & Montes Gan, Vicente J & Moreno-Brieva, Fernando, 2025. "Non-hierarchic leadership collaboration: Exploring the adoption of AI-driven social networking for addressing social challenges in an extra-organizational environment," Technology in Society, Elsevier, vol. 81(C).
    9. Messono, Omang Ombolo, 2025. "The intersection of publics services digitalization and women's empowerment in tax revenue mobilization," Technological Forecasting and Social Change, Elsevier, vol. 219(C).
    10. Hung, Chih-Lung & Wu, Jen-Her & Chiang, Po-Chuen & Li, Qi & Chen, Yi-Cheng, 2025. "Factors driving user behavior and value creation with text-to-image generative artificial intelligence (AI): A systems theory perspective," Technology in Society, Elsevier, vol. 83(C).
    11. Abigail Devereaux, 2025. "Hot nudges on hazy landscapes," Mind & Society: Cognitive Studies in Economics and Social Sciences, Springer;Fondazione Rosselli, vol. 24(2), pages 795-826, December.
    12. Deyin Zhang & Shijian Luo & Yue Liu & Xiyuan Zhang & Yuqi Hu & Jianan Shen & Peiqi Yi & Kayu Zhao & Wei Liu, 2026. "Large language model tools as catalysts for collective cognition in collaborative new-product development: a quasi-experimental study," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 13(1), pages 1-20, December.
    13. Hrvoje Carić & Ante Mandić & Ivan Sever, 2026. "A six-phase AI-expert framework for evaluating policy coherence in sustainable tourism," Information Technology & Tourism, Springer, vol. 28(1), pages 1-30, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pietronudo, Maria Cristina & Croidieu, Grégoire & Schiavone, Francesco, 2022. "A solution looking for problems? A systematic literature review of the rationalizing influence of artificial intelligence on decision-making in innovation management," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    2. Stefan Scherbaum & Steven J Lade & Stefan Siegmund & Thomas Goschke & Maja Dshemuchadse, 2022. "From single decisions to sequential choice patterns: Extending the dynamics of value-based decision-making," PLOS ONE, Public Library of Science, vol. 17(4), pages 1-16, April.
    3. António Madureira & Nico Baken & Harry Bouwman, 2011. "Value of digital information networks: a holonic framework," Netnomics, Springer, vol. 12(1), pages 1-30, April.
    4. Rice, Stephen & Crouse, Sean R. & Winter, Scott R. & Rice, Connor, 2024. "The advantages and limitations of using ChatGPT to enhance technological research," Technology in Society, Elsevier, vol. 76(C).
    5. Lena Nadarevic & Martin Schnuerch & Marlena J. Stegemann, 2021. "Judging fast and slow: The truth effect does not increase under time-pressure conditions," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 16(5), pages 1234-1266, September.
    6. Richard Iles & Haniel Gatumu & Samuel Kagunda, 2019. "The role of poverty on economic decision-making: a model of cognitive function and heuristic use," Working Papers 2019-3, School of Economic Sciences, Washington State University.
    7. repec:plo:pone00:0241182 is not listed on IDEAS
    8. Andreas Glöckner & Stephan Dickert, 2008. "Base-rate Respect by Intuition: Approximating Rational Choices in Base-rate Tasks with Multiple Cues," Discussion Paper Series of the Max Planck Institute for Behavioral Economics 2008_49, Max Planck Institute for Behavioral Economics.
    9. Christoph Engel & Andreas Glöckner, 2008. "Can We Trust Intuitive Jurors? An Experimental Analysis," Discussion Paper Series of the Max Planck Institute for Behavioral Economics 2008_36, Max Planck Institute for Behavioral Economics.
    10. Christoph Engel, 2010. "The Multiple Uses of Experimental Evidence in Legal Scholarship," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 166(1), pages 199-202, March.
    11. Julian N. Marewski & Katja Mehlhorn, 2011. "Using the ACT-R architecture to specify 39 quantitative process models of decision making," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 6(6), pages 439-519, August.
    12. Ben Newell & Arndt Bröder, 2008. "Cognitive processes, models and metaphors in decision research," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 3, pages 195-204, March.
    13. Julian N. Marewski & Rudiger F. Pohl & Oliver Vitouch, 2011. "Recognition-based judgments and decisions: Introduction to the special issue (II)," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 6(1), pages 1-6, February.
    14. Osita Amaugo, 2025. "Impact of Artificial Intelligence on Effective Decision Making in Corporate Financial Entities in Nigeria (A Case Study of Fidelity Bank)," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 9(5), pages 2407-2416, May.
    15. Giustina Secundo & Claudia Spilotro & Johanna Gast & Vincenzo Corvello, 2025. "The transformative power of artificial intelligence within innovation ecosystems: a review and a conceptual framework," Review of Managerial Science, Springer, vol. 19(9), pages 2697-2728, September.
    16. A. Madureira & F. Hartog & N. Baken, 2016. "A holonic framework to understand and apply information processes in evolutionary economics: survey and proposal," Netnomics, Springer, vol. 17(2), pages 157-190, September.
    17. Volkmar, Gioia & Fischer, Peter M. & Reinecke, Sven, 2022. "Artificial Intelligence and Machine Learning: Exploring drivers, barriers, and future developments in marketing management," Journal of Business Research, Elsevier, vol. 149(C), pages 599-614.
    18. Tielman, Myrthe L. & Suárez-Figueroa, Mari Carmen & Jönsson, Arne & Neerincx, Mark A. & Cavalcante Siebert, Luciano, 2024. "Explainable AI for all - A roadmap for inclusive XAI for people with cognitive disabilities," Technology in Society, Elsevier, vol. 79(C).
    19. Anne-Sophie Chaxel & J. Edward Russo & Neda Kerimi, 2013. "Preference-driven biases in decision makers' information search and evaluation," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 8(5), pages 561-576, September.
    20. Zhao, Xuefeng & Wu, Weiwei & Wu, Delin, 2024. "How does AI perform in industry chain? A patent claims analysis approach," Technology in Society, Elsevier, vol. 79(C).
    21. Charles, Vincent & Emrouznejad, Ali & Kunz, Werner H., 2025. "Advancements in Artificial Intelligence-based prescriptive and cognitive analytics for business performance: a special issue editorial," Journal of Business Research, Elsevier, vol. 200(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:teinso:v:78:y:2024:i:c:s0160791x24002100. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/technology-in-society .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.