IDEAS home Printed from https://ideas.repec.org/a/eee/jbrese/v199y2025ics0148296325003947.html

Avoiding algorithm errors in textual analysis: A guide to selecting software, and a research agenda toward generative artificial intelligence

Author

Listed:
  • Wobst, Janice
  • Lueg, Rainer

Abstract

The use of textual analysis is expanding in organizational research, yet software packages vary in their compatibility with complex constructs. This study helps researchers select suitable tools by focusing on phrase-based dictionary methods. We empirically evaluate four software packages—LIWC, DICTION, CAT Scanner, and a custom Python tool—using the complex construct of value-based management as a test case. The analysis shows that software from the same methodological family produces highly consistent results, while popular but mismatched tools yield significant errors such as miscounted phrases. Based on this, we develop a structured selection guideline that links construct features with software capabilities. The framework enhances construct validity, supports methodological transparency, and is applicable across disciplines. Finally, we position the approach as a bridge to AI-enabled textual analysis, including prompt-based workflows, reinforcing the continued need for theory-grounded construct design.

Suggested Citation

  • Wobst, Janice & Lueg, Rainer, 2025. "Avoiding algorithm errors in textual analysis: A guide to selecting software, and a research agenda toward generative artificial intelligence," Journal of Business Research, Elsevier, vol. 199(C).
    • Handle: RePEc:eee:jbrese:v:199:y:2025:i:c:s0148296325003947
    DOI: 10.1016/j.jbusres.2025.115571
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0148296325003947
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jbusres.2025.115571?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Rezazadeh, Arash & Kohns, Marco & Bohnsack, René & António, Nuno & Rita, Paulo, 2025. "Generative AI for growth hacking: How startups use generative AI in their growth strategies," Journal of Business Research, Elsevier, vol. 192(C).
    2. Paul C. Godfrey & Charles W. L. Hill, 1995. "The problem of unobservables in strategic management research," Strategic Management Journal, Wiley Blackwell, vol. 16(7), pages 519-533.
    3. Ittner, Christopher D. & Larcker, David F., 2001. "Assessing empirical research in managerial accounting: a value-based management perspective," Journal of Accounting and Economics, Elsevier, vol. 32(1-3), pages 349-410, December.
    4. Kirsten Martin, 2019. "Ethical Implications and Accountability of Algorithms," Journal of Business Ethics, Springer, vol. 160(4), pages 835-850, December.
    5. Gaur, Ajai & Kumar, Mukesh, 2018. "A systematic approach to conducting review studies: An assessment of content analysis in 25years of IB research," Journal of World Business, Elsevier, vol. 53(2), pages 280-289.
    6. Evert, Robert E. & Payne, G. Tyge & Moore, Curt B. & McLeod, Michael S., 2018. "Top Management Team Characteristics and Organizational Virtue Orientation: An Empirical Examination of IPO Firms," Business Ethics Quarterly, Cambridge University Press, vol. 28(4), pages 427-461, October.
    7. Sebastian Firk & Torben Schmidt & Michael Wolff, 2019. "Exploring Value‐Based Management Sophistication: The Role of Potential Economic Benefits and Institutional Influence," Contemporary Accounting Research, John Wiley & Sons, vol. 36(1), pages 418-450, March.
    8. Tim Loughran & Bill Mcdonald, 2016. "Textual Analysis in Accounting and Finance: A Survey," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 54(4), pages 1187-1230, September.
    9. Chan, Hau-Ling & Choi, Tsan-Ming, 2025. "Using generative artificial intelligence (GenAI) in marketing: Development and practices," Journal of Business Research, Elsevier, vol. 191(C).
    10. Mariani, Marcello & Dwivedi, Yogesh K., 2024. "Generative artificial intelligence in innovation management: A preview of future research developments," Journal of Business Research, Elsevier, vol. 175(C).
    11. Anglin, Aaron H. & Wolfe, Marcus T. & Short, Jeremy C. & McKenny, Aaron F. & Pidduck, Robert J., 2018. "Narcissistic rhetoric and crowdfunding performance: A social role theory perspective," Journal of Business Venturing, Elsevier, vol. 33(6), pages 780-812.
    12. Hermann, Erik & Puntoni, Stefano, 2024. "Artificial intelligence and consumer behavior: From predictive to generative AI," Journal of Business Research, Elsevier, vol. 180(C).
    13. David A. Hull, 1996. "Stemming algorithms: A case study for detailed evaluation," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 47(1), pages 70-84, January.
    14. Herhausen, Dennis & Ludwig, Stephan & Abedin, Ehsan & Haque, Nasim Ul & de Jong, David, 2025. "From words to insights: Text analysis in business research," Journal of Business Research, Elsevier, vol. 198(C).
    15. Mansouri, Sasan & Momtaz, Paul P., 2022. "Financing sustainable entrepreneurship: ESG measurement, valuation, and performance," Journal of Business Venturing, Elsevier, vol. 37(6).
    16. Bisbe, Josep & Batista-Foguet, Joan-Manuel & Chenhall, Robert, 2007. "Defining management accounting constructs: A methodological note on the risks of conceptual misspecification," Accounting, Organizations and Society, Elsevier, vol. 32(7-8), pages 789-820.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Thomas, Llewellyn D.W. & Romasanta, Angelo Kenneth G. & Pujol Priego, Laia, 2026. "Jagged competencies: Measuring the reliability of generative AI in academic research," Journal of Business Research, Elsevier, vol. 203(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christoph Feichter & Isabella Grabner, 2020. "Empirische Forschung zu Management Control – Ein Überblick und neue Trends [Empirical Management Control Reserach—An Overview and Future Directions]," Schmalenbach Journal of Business Research, Springer, vol. 72(2), pages 149-181, June.
    2. Anna Kusetogullari & Huseyin Kusetogullari & Martin Andersson & Tony Gorschek, 2025. "GenAI in Entrepreneurship: a systematic review of generative artificial intelligence in entrepreneurship research: current issues and future directions," Papers 2505.05523, arXiv.org.
    3. Karim Mahran & Ahmed A. Elamer, 2026. "Navigating the Corporate Ego: Understanding the Association Between ESG Performance and Organizational Narcissistic Rhetoric," Journal of Business Ethics, Springer, vol. 204(2), pages 437-456, March.
    4. Giacomo Zatini, 2025. "Conditions of use and impacts of artificial intelligence in marketing practices: a mixed-method literature review," Italian Journal of Marketing, Springer, vol. 2025(3), pages 293-329, September.
    5. Michael S. McLeod & Joshua B. Sears & Gaylen N. Chandler & G. Tyge Payne & Keith H. Brigham, 2022. "Rhetoric, Risk, and Investment: Letting the Numbers Speak for Themselves," Journal of Management Studies, Wiley Blackwell, vol. 59(7), pages 1657-1687, November.
    6. Arnold, Markus C. & Artz, Martin, 2015. "Target difficulty, target flexibility, and firm performance: Evidence from business units’ targets," Accounting, Organizations and Society, Elsevier, vol. 40(C), pages 61-77.
    7. Abernethy, Margaret A. & Bouwens, Jan & Kroos, Peter, 2017. "Organization identity and earnings manipulation," Accounting, Organizations and Society, Elsevier, vol. 58(C), pages 1-14.
    8. Xiaoyue Lin & Tiandong Wang & Fan Sheng, 2025. "Exploring the dual effect of trust in GAI on employees’ exploitative and exploratory innovation," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-14, December.
    9. Chan, Hau-Ling & Choi, Tsan-Ming, 2025. "Using generative artificial intelligence (GenAI) in marketing: Development and practices," Journal of Business Research, Elsevier, vol. 191(C).
    10. Franco Visani, 2017. "Applying business analytics for performance measurement and management. The case study of a software company," MANAGEMENT CONTROL, FrancoAngeli Editore, vol. 2017(2), pages 89-123.
    11. Aernoudts, R.H.R.M. & De Heer, M.A., 2008. "On the Interplay between Strategy and Management Control Systems," MPRA Paper 9583, University Library of Munich, Germany.
    12. Mark R. DesJardine & Wei Shi, 2023. "The Downside of Displaying Agentic Values: Evidence from Shareholder Activism," Organization Science, INFORMS, vol. 34(4), pages 1577-1601, July.
    13. Demartini, Maria Chiara & Otley, David, 2020. "Beyond the system vs. package dualism in Performance Management Systems design: A loose coupling approach," Accounting, Organizations and Society, Elsevier, vol. 86(C).
    14. Karim Mahran & Ahmed A. Elamer, 2025. "Shaping ESG commitment through organizational psychological capital: The role of CEO power," Business Strategy and the Environment, Wiley Blackwell, vol. 34(1), pages 690-707, January.
    15. Marina Dabić & Jane Maley & Leo-Paul Dana & Ivan Novak & Massimiliano M. Pellegrini & Andrea Caputo, 2020. "Pathways of SME internationalization: a bibliometric and systematic review," Small Business Economics, Springer, vol. 55(3), pages 705-725, October.
    16. Niyi Ogunbiyi & Artie Basukoski & Thierry Chaussalet, 2021. "An Exploration of Ethical Decision Making with Intelligence Augmentation," Social Sciences, MDPI, vol. 10(2), pages 1-14, February.
    17. Wolfgang Drobetz & Lars Hornuf & Paul P. Momtaz & Niclas Schermann, 2025. "Token-Based Crowdfunding: Investor Choice and the Optimal Timing of Initial Coin Offerings," Entrepreneurship Theory and Practice, , vol. 49(1), pages 232-282, January.
    18. Utz Schäffer & Matthias D. Mahlendorf & Jochen Rehring, 2014. "Does the Interactive Use of Headquarter Performance Measurement Systems in Foreign Subsidiaries Endanger the Potential to Profit from Local Relationships?," Australian Accounting Review, CPA Australia, vol. 24(1), pages 21-38, March.
    19. Oehler, Andreas & Neuss, Charlotte, 2025. "ESG disclosure vs. ESG ratings: Consistent information value?," International Review of Financial Analysis, Elsevier, vol. 107(C).
    20. Yang, Jialiang & Li, Yaokuang & Calic, Goran & Shevchenko, Anton, 2020. "How multimedia shape crowdfunding outcomes: The overshadowing effect of images and videos on text in campaign information," Journal of Business Research, Elsevier, vol. 117(C), pages 6-18.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C80 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - General
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software
    • M10 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - General
    • M15 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - IT Management
    • L86 - Industrial Organization - - Industry Studies: Services - - - Information and Internet Services; Computer Software

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jbrese:v:199:y:2025:i:c:s0148296325003947. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jbusres .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.