IDEAS home Printed from https://ideas.repec.org/a/bla/jinfst/v73y2022i12p1755-1775.html
   My bibliography  Save this article

Analysis of noise and bias errors in intelligence information systems

Author

Listed:
  • Ashraf Labib
  • Salem Chakhar
  • Lorraine Hope
  • John Shimell
  • Mark Malinowski

Abstract

An intelligence information system (IIS) is a particular kind of information systems (IS) devoted to the analysis of intelligence relevant to national security. Professional and military intelligence analysts play a key role in this, but their judgments can be inconsistent, mainly due to noise and bias. The team‐oriented aspects of the intelligence analysis process complicates the situation further. To enable analysts to achieve better judgments, the authors designed, implemented, and validated an innovative IIS for analyzing UK Military Signals Intelligence (SIGINT) data. The developed tool, the Team Information Decision Engine (TIDE), relies on an innovative preference learning method along with an aggregation procedure that permits combining scores by individual analysts into aggregated scores. This paper reports on a series of validation trials in which the performance of individual and team‐oriented analysts was accessed with respect to their effectiveness and efficiency. Results show that the use of the developed tool enhanced the effectiveness and efficiency of intelligence analysis process at both individual and team levels.

Suggested Citation

  • Ashraf Labib & Salem Chakhar & Lorraine Hope & John Shimell & Mark Malinowski, 2022. "Analysis of noise and bias errors in intelligence information systems," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(12), pages 1755-1775, December.
    • Handle: RePEc:bla:jinfst:v:73:y:2022:i:12:p:1755-1775
    DOI: 10.1002/asi.24707
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.24707
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.24707?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Greco, Salvatore & Matarazzo, Benedetto & Slowinski, Roman, 2001. "Rough sets theory for multicriteria decision analysis," European Journal of Operational Research, Elsevier, vol. 129(1), pages 1-47, February.
    2. Omri Suissa & Avshalom Elmalech & Maayan Zhitomirsky‐Geffet, 2022. "Text analysis using deep neural networks in digital humanities and information science," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(2), pages 268-287, February.
    3. David Allen & Lisa M. Given & Gary Burnett & Stan Karanasios, 2019. "Information behavior and information practices: A special issue for research on people's engagement with technology," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 70(12), pages 1299-1301, December.
    4. Bruno Biais & Martin Weber, 2009. "Hindsight Bias, Risk Perception, and Investment Performance," Management Science, INFORMS, vol. 55(6), pages 1018-1029, June.
    5. Anton van der Vegt & Guido Zuccon & Bevan Koopman, 2021. "Do better search engines really equate to better clinical decisions? If not, why not?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(2), pages 141-155, February.
    6. David Allen, 2011. "Information behavior and decision making in time‐constrained practice: A dual‐processing perspective," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(11), pages 2165-2181, November.
    7. Hong Hong & Qiang Ye & Qianzhou Du & G. Alan Wang & Weiguo Fan, 2020. "Crowd characteristics and crowd wisdom: Evidence from an online investment community," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(4), pages 423-435, April.
    8. W. John Wilbur, 1998. "A comparison of group and individual performance among subject experts and untrained workers at the document retrieval task," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 49(6), pages 517-529.
    9. Gediminas Adomavicius & Alok Gupta & Dmitry Zhdanov, 2009. "Designing Intelligent Software Agents for Auctions with Limited Information Feedback," Information Systems Research, INFORMS, vol. 20(4), pages 507-526, December.
    10. Song-Hee Kim & Jordan Tong & Carol Peden, 2020. "Admission Control Biases in Hospital Unit Capacity Management: How Occupancy Information Hurdles and Decision Noise Impact Utilization," Management Science, INFORMS, vol. 66(11), pages 5151-5170, November.
    11. Hongjun Yan, 2010. "Is Noise Trading Cancelled Out by Aggregation?," Management Science, INFORMS, vol. 56(7), pages 1047-1059, July.
    12. Alex Bordetsky & Gloria Mark, 2000. "Memory-Based Feedback Controls to Support Groupware Coordination," Information Systems Research, INFORMS, vol. 11(4), pages 366-385, December.
    13. Gilberto Montibeller & Detlof Winterfeldt, 2018. "Individual and Group Biases in Value and Uncertainty Judgments," International Series in Operations Research & Management Science, in: Luis C. Dias & Alec Morton & John Quigley (ed.), Elicitation, chapter 0, pages 377-392, Springer.
    14. William Yeoh & Aleš Popovič, 2016. "Extending the understanding of critical success factors for implementing business intelligence systems," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(1), pages 134-147, January.
    15. Shleifer, Andrei, 2000. "Inefficient Markets: An Introduction to Behavioral Finance," OUP Catalogue, Oxford University Press, number 9780198292272, Decembrie.
    16. Gilberto Montibeller & Detlof von Winterfeldt, 2015. "Cognitive and Motivational Biases in Decision and Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 35(7), pages 1230-1251, July.
    17. Jyoti Mishra & David Allen & Alan Pearman, 2015. "Information seeking, use, and decision making," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(4), pages 662-673, April.
    18. Qian Liu & Zhengfa Yang & Xiaofang Cai & Qianzhou Du & Weiguo Fan, 2022. "The more, the better? The effect of feedback and user's past successes on idea implementation in open innovation communities," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(3), pages 376-392, March.
    19. Lionel P. Robert Jr & Alan R. Dennis & Manju K. Ahuja, 2018. "Differences are Different: Examining the Effects of Communication Media on the Impacts of Racial and Gender Diversity in Decision-Making Teams," Information Systems Research, INFORMS, vol. 29(3), pages 525-545, September.
    20. Chakhar, Salem & Ishizaka, Alessio & Labib, Ashraf & Saad, Inès, 2016. "Dominance-based rough set approach for group decisions," European Journal of Operational Research, Elsevier, vol. 251(1), pages 206-224.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Volker Thoma & Elliott White & Asha Panigrahi & Vanessa Strowger & Irina Anderson, 2015. "Good Thinking or Gut Feeling? Cognitive Reflection and Intuition in Traders, Bankers and Financial Non-Experts," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-17, April.
    2. Azam, Nouman & Zhang, Yan & Yao, JingTao, 2017. "Evaluation functions and decision conditions of three-way decisions with game-theoretic rough sets," European Journal of Operational Research, Elsevier, vol. 261(2), pages 704-714.
    3. Hu, Qiwei & Chakhar, Salem & Siraj, Sajid & Labib, Ashraf, 2017. "Spare parts classification in industrial manufacturing using the dominance-based rough set approach," European Journal of Operational Research, Elsevier, vol. 262(3), pages 1136-1163.
    4. Julio Cezar Soares Silva & Diogo Ferreira de Lima Silva & Luciano Ferreira & Adiel Teixeira de Almeida-Filho, 2022. "A dominance-based rough set approach applied to evaluate the credit risk of sovereign bonds," 4OR, Springer, vol. 20(1), pages 139-164, March.
    5. Fatima‐Zohra Younsi & Salem Chakhar & Alessio Ishizaka & Djamila Hamdadou & Omar Boussaid, 2020. "A Dominance‐Based Rough Set Approach for an Enhanced Assessment of Seasonal Influenza Risk," Risk Analysis, John Wiley & Sons, vol. 40(7), pages 1323-1341, July.
    6. Raman Uppal & Harjoat Bhamra, 2016. "Do Individual Behavioral Biases Affect Financial Markets and the Macroeconomy?," 2016 Meeting Papers 1358, Society for Economic Dynamics.
    7. Anderson, Irina & Thoma, Volker, 2021. "The edge of reason: A thematic analysis of how professional financial traders understand analytical decision making," European Management Journal, Elsevier, vol. 39(2), pages 304-314.
    8. Du, Wen Sheng & Hu, Bao Qing, 2018. "A fast heuristic attribute reduction approach to ordered decision systems," European Journal of Operational Research, Elsevier, vol. 264(2), pages 440-452.
    9. Miłosz Kadziński & Lucia Rocchi & Grzegorz Miebs & David Grohmann & Maria Elena Menconi & Luisa Paolotti, 2018. "Multiple Criteria Assessment of Insulating Materials with a Group Decision Framework Incorporating Outranking Preference Model and Characteristic Class Profiles," Group Decision and Negotiation, Springer, vol. 27(1), pages 33-59, February.
    10. Bouzayane, Sarra & Saad, Inès, 2020. "A multicriteria approach based on rough set theory for the incremental Periodic prediction," European Journal of Operational Research, Elsevier, vol. 286(1), pages 282-298.
    11. Kuller, M. & Beutler, P. & Lienert, J., 2023. "Preference change in stakeholder group-decision processes in the public sector: Extent, causes and implications," European Journal of Operational Research, Elsevier, vol. 308(3), pages 1268-1285.
    12. Li, Jinfang, 2017. "Investor sentiment, heterogeneous agents and asset pricing model," The North American Journal of Economics and Finance, Elsevier, vol. 42(C), pages 504-512.
    13. Robin L. Dillon & Vicki M. Bier & Richard Sheffield John & Abdullah Althenayyan, 2023. "Closing the Gap Between Decision Analysis and Policy Analysts Before the Next Pandemic," Decision Analysis, INFORMS, vol. 20(2), pages 109-132, June.
    14. Ben Krishna & Anindita Paul, 2020. "Collaborative information behaviour during epidemics: The case of Nipah outbreak in Southern India," Working papers 367, Indian Institute of Management Kozhikode.
    15. Gilberto Montibeller & L. Alberto Franco & Ashley Carreras, 2020. "A Risk Analysis Framework for Prioritizing and Managing Biosecurity Threats," Risk Analysis, John Wiley & Sons, vol. 40(11), pages 2462-2477, November.
    16. Chakhar, Salem & Ishizaka, Alessio & Thorpe, Andy & Cox, Joe & Nguyen, Thang & Ford, Liz, 2020. "Calculating the relative importance of condition attributes based on the characteristics of decision rules and attribute reducts: Application to crowdfunding," European Journal of Operational Research, Elsevier, vol. 286(2), pages 689-712.
    17. Mota, Caroline Maria de Miranda & Figueiredo, Ciro José Jardim de & Pereira, Débora Viana e Sousa, 2021. "Identifying areas vulnerable to homicide using multiple criteria analysis and spatial analysis," Omega, Elsevier, vol. 100(C).
    18. Du, Wen Sheng & Hu, Bao Qing, 2017. "Dominance-based rough fuzzy set approach and its application to rule induction," European Journal of Operational Research, Elsevier, vol. 261(2), pages 690-703.
    19. Huang, Bing & Li, Huaxiong & Feng, Guofu & Zhou, Xianzhong, 2019. "Dominance-based rough sets in multi-scale intuitionistic fuzzy decision tables," Applied Mathematics and Computation, Elsevier, vol. 348(C), pages 487-512.
    20. Haag, Fridolin & Zürcher, Sara & Lienert, Judit, 2019. "Enhancing the elicitation of diverse decision objectives for public planning," European Journal of Operational Research, Elsevier, vol. 279(3), pages 912-928.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jinfst:v:73:y:2022:i:12:p:1755-1775. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.