IDEAS home Printed from https://ideas.repec.org/a/wly/fufsci/v3y2021i1ne56.html
   My bibliography  Save this article

Improving the question formulation in Delphi‐like surveys: Analysis of the effects of abstract language and amount of information on response behavior

Author

Listed:
  • Christoph Markmann
  • Alexander Spickermann
  • Heiko A. von der Gracht
  • Alexander Brem

Abstract

Prospective surveys are a common research mode used to establish a profound decision base for R&D investments, technological innovation, and policy making. The challenge faced by participants of such surveys, particularly if conducted via impersonal survey modes (e.g., mail or online surveys), is the limited ability to clarify contextual issues. Related research in socio‐psychology refers to language vagueness as abstractness, cautioning that abstract language and the way people interpret information are related. We apply Construal Level Theory and the Linguistic Category Model from linguistic sciences to the context of prospective surveys and examine the effects of abstract language and its related amount of information on panelists' response behavior. Based upon 32,000 observations, our research reveals that higher abstractness and greater amount of information reduce the clarity of the respondents' assessments of survey questions. In particular, abstract language and lengthy questions lead to less extreme assessments and a reduced response variance of panelists.

Suggested Citation

  • Christoph Markmann & Alexander Spickermann & Heiko A. von der Gracht & Alexander Brem, 2021. "Improving the question formulation in Delphi‐like surveys: Analysis of the effects of abstract language and amount of information on response behavior," Futures & Foresight Science, John Wiley & Sons, vol. 3(1), March.
    • Handle: RePEc:wly:fufsci:v:3:y:2021:i:1:n:e56
    DOI: 10.1002/ffo2.56
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/ffo2.56
    Download Restriction: no
    File URL: https://libkey.io/10.1002/ffo2.56?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Nils Plambeck & K. Weber, 2009. "CEO Ambivalence and Responses to Strategic Issues," Post-Print hal-00464009, HAL.
    2. Haegeman, Karel & Marinelli, Elisabetta & Scapolo, Fabiana & Ricci, Andrea & Sokolov, Alexander, 2013. "Quantitative and qualitative approaches in Future-oriented Technology Analysis (FTA): From combination to integration?," Technological Forecasting and Social Change, Elsevier, vol. 80(3), pages 386-397.
    3. Gerard Stone & Lee D. Parker, 2013. "Developing the Flesch reading ease formula for the contemporary accounting communications landscape," Qualitative Research in Accounting & Management, Emerald Group Publishing Limited, vol. 10(1), pages 31-59, April.
    4. Ravonne A. Green, 2014. "The Delphi Technique in Educational Research," SAGE Open, , vol. 4(2), pages 21582440145, April.
    5. Graefe, Andreas & Armstrong, J. Scott, 2011. "Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task," International Journal of Forecasting, Elsevier, vol. 27(1), pages 183-195, January.
    6. Kuusi, Osmo, 1999. "Expertise in the Future Use of Generic Technologies," Research Reports 59, VATT Institute for Economic Research.
    7. Vecchiato, Riccardo, 2012. "Environmental uncertainty, foresight and strategic decision making: An integrated study," Technological Forecasting and Social Change, Elsevier, vol. 79(3), pages 436-447.
    8. Akkermans, Henk A. & Bogerd, Paul & Yucesan, Enver & van Wassenhove, Luk N., 2003. "The impact of ERP on supply chain management: Exploratory findings from a European Delphi study," European Journal of Operational Research, Elsevier, vol. 146(2), pages 284-301, April.
    9. Michael R Czinkota & Ilkka A Ronkainen, 1997. "International Business and Trade in the Next Decade: Report from a Delphi Study," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 28(4), pages 827-844, December.
    10. Gerard Stone & Lee D. Parker, 2013. "Developing the Flesch reading ease formula for the contemporary accounting communications landscape," Qualitative Research in Accounting & Management, Emerald Group Publishing Limited, vol. 10(1), pages 31-59, April.
    11. Tihanyi, Laszlo & Thomas, Wayne B., 2005. "Information-processing demands and the multinational enterprise: a comparison of foreign and domestic earnings estimates," Journal of Business Research, Elsevier, vol. 58(3), pages 285-292, March.
    12. Saviotti, Pier Paolo, 1998. "On the dynamics of appropriability, of tacit and of codified knowledge," Research Policy, Elsevier, vol. 26(7-8), pages 843-856, April.
    13. Buesa, Mikel & Heijs, Joost & Baumert, Thomas, 2010. "The determinants of regional innovation in Europe: A combined factorial and regression knowledge production function approach," Research Policy, Elsevier, vol. 39(6), pages 722-735, July.
    14. Hsiao, Tzy-yih, 2006. "Establish standards of standard costing with the application of convergent gray zone test," European Journal of Operational Research, Elsevier, vol. 168(2), pages 593-611, January.
    15. Kowalczyk, Martin & Buxmann, Peter, 2015. "An Ambidextrous Perspective on Business Intelligence and Analytics Support in Decision Processes: Insights from a Multiple Case Study," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 75107, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    16. Philippe Petithuguenin & Marie de Lattre-Gasquet & Jérôme Sainte-Beuve, 2003. "Foresight in a research institution: a critical review of two exercises," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 22(2-3), pages 203-217.
    17. Georghiou, Luke & Cassingena Harper, Jennifer, 2013. "Rising to the challenges—Reflections on Future-oriented Technology Analysis," Technological Forecasting and Social Change, Elsevier, vol. 80(3), pages 467-470.
    18. Munier, Francis & Ronde, Patrick, 2001. "The role of knowledge codification in the emergence of consensus under uncertainty: empirical analysis and policy implications," Research Policy, Elsevier, vol. 30(9), pages 1537-1551, December.
    19. Rajshree Agarwal & Constance E. Helfat, 2009. "Strategic Renewal of Organizations," Organization Science, INFORMS, vol. 20(2), pages 281-293, April.
    20. Nils Plambeck & Klaus Weber, 2010. "When the Glass is Half Full and Half Empty: CEO's Ambivalent Interpretations of Strategic Issues," Post-Print hal-00528394, HAL.
    21. Meijering, J.V. & Kampen, J.K. & Tobi, H., 2013. "Quantifying the development of agreement among experts in Delphi studies," Technological Forecasting and Social Change, Elsevier, vol. 80(8), pages 1607-1614.
    22. Sauermann, Henry & Roach, Michael, 2013. "Increasing web survey response rates in innovation research: An experimental study of static and dynamic contact design features," Research Policy, Elsevier, vol. 42(1), pages 273-286.
    23. Broström, Anders, 2010. "Working with distant researchers--Distance and content in university-industry interaction," Research Policy, Elsevier, vol. 39(10), pages 1311-1320, December.
    24. Lazaric, Nathalie & Mangolte, Pierre-Andre & Massue, Marie-Laure, 2003. "Articulation and codification of collective know-how in the steel industry: evidence from blast furnace control in France," Research Policy, Elsevier, vol. 32(10), pages 1829-1847, December.
    25. Haeussler, Carolin, 2011. "Information-sharing in academia and the industry: A comparative study," Research Policy, Elsevier, vol. 40(1), pages 105-122, February.
    26. Geist, Monica R., 2010. "Using the Delphi method to engage stakeholders: A comparison of two studies," Evaluation and Program Planning, Elsevier, vol. 33(2), pages 147-154, May.
    27. Morlacchi, Piera & Martin, Ben R., 2009. "Emerging challenges for science, technology and innovation policy research: A reflexive overview," Research Policy, Elsevier, vol. 38(4), pages 571-582, May.
    28. Bokrantz, Jon & Skoogh, Anders & Berlin, Cecilia & Stahre, Johan, 2017. "Maintenance in digitalised manufacturing: Delphi-based scenarios for 2030," International Journal of Production Economics, Elsevier, vol. 191(C), pages 154-169.
    29. Leopold Bayerlein & Paul Davidson, 2011. "The influence of connotation on readability and obfuscation in Australian chairman addresses," Managerial Auditing Journal, Emerald Group Publishing, vol. 27(2), pages 175-198, December.
    30. Rohrbeck, René & Schwarz, Jan Oliver, 2013. "The value contribution of strategic foresight: Insights from an empirical study of large European companies," Technological Forecasting and Social Change, Elsevier, vol. 80(8), pages 1593-1606.
    31. Markard, Jochen & Stadelmann, Martin & Truffer, Bernhard, 2009. "Prospective analysis of technological innovation systems: Identifying technological and organizational development options for biogas in Switzerland," Research Policy, Elsevier, vol. 38(4), pages 655-667, May.
    32. Nils Plambeck & Klaus Weber, 2010. "When the glass is half full and half empty: CEOs' ambivalent interpretations of strategic issues," Strategic Management Journal, Wiley Blackwell, vol. 31(7), pages 689-710, July.
    33. Wiesenthal, Tobias & Leduc, Guillaume & Haegeman, Karel & Schwarz, Hans-Günther, 2012. "Bottom-up estimation of industrial and public R&D investment by technology in support of policy-making: The case of selected low-carbon energy technologies," Research Policy, Elsevier, vol. 41(1), pages 116-131.
    34. Nill, Jan & Kemp, Ren, 2009. "Evolutionary approaches for sustainable innovation policies: From niche to paradigm?," Research Policy, Elsevier, vol. 38(4), pages 668-680, May.
    35. Winkler, Jens & Moser, Roger, 2016. "Biases in future-oriented Delphi studies: A cognitive perspective," Technological Forecasting and Social Change, Elsevier, vol. 105(C), pages 63-76.
    36. von der Gracht, Heiko A., 2012. "Consensus measurement in Delphi studies," Technological Forecasting and Social Change, Elsevier, vol. 79(8), pages 1525-1536.
    37. Markard, Jochen & Raven, Rob & Truffer, Bernhard, 2012. "Sustainability transitions: An emerging field of research and its prospects," Research Policy, Elsevier, vol. 41(6), pages 955-967.
    38. Todtling, Franz & Trippl, Michaela, 2005. "One size fits all?: Towards a differentiated regional innovation policy approach," Research Policy, Elsevier, vol. 34(8), pages 1203-1219, October.
    39. Philipp Ecken & Richard Pibernik, 2016. "Hit or Miss: What Leads Experts to Take Advice for Long-Term Judgments?," Management Science, INFORMS, vol. 62(7), pages 2002-2021, July.
    40. Bruine de Bruin, Wändi & van der Klaauw, Wilbert & Topa, Giorgio & Downs, Julie S. & Fischhoff, Baruch & Armantier, Olivier, 2012. "The effect of question wording on consumers’ reported inflation expectations," Journal of Economic Psychology, Elsevier, vol. 33(4), pages 749-757.
    41. Calabor, Mari Sol & Mora, Araceli & Moya, Soledad, 2019. "The future of 'serious games' in accounting education: A Delphi study," Journal of Accounting Education, Elsevier, vol. 46(C), pages 43-52.
    42. Bonaccorsi, Andrea & Apreda, Riccardo & Fantoni, Gualtiero, 2020. "Expert biases in technology foresight. Why they are a problem and how to mitigate them," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    43. Mauksch, Stefanie & von der Gracht, Heiko A. & Gordon, Theodore J., 2020. "Who is an expert for foresight? A review of identification methods," Technological Forecasting and Social Change, Elsevier, vol. 154(C).
    44. Norman Dalkey & Olaf Helmer, 1963. "An Experimental Application of the DELPHI Method to the Use of Experts," Management Science, INFORMS, vol. 9(3), pages 458-467, April.
    45. Nils Plambeck & Klaus Weber, 2009. "CEO Ambivalence and Responses to Strategic Issues," Organization Science, INFORMS, vol. 20(6), pages 993-1010, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Haidar, Ali & Guimón, José & Alon, Ido, 2024. "Can graphene fuel a transformative change in energy storage technologies? A scenario analysis for the next two decades," Technological Forecasting and Social Change, Elsevier, vol. 202(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tiberius, Victor & Gojowy, Robin & Dabić, Marina, 2022. "Forecasting the future of robo advisory: A three-stage Delphi study on economic, technological, and societal implications," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    2. Peppel, Marcel & Ringbeck, Jürgen & Spinler, Stefan, 2022. "How will last-mile delivery be shaped in 2040? A Delphi-based scenario study," Technological Forecasting and Social Change, Elsevier, vol. 177(C).
    3. Frevel, Nicolas & Beiderbeck, Daniel & Schmidt, Sascha L., 2022. "The impact of technology on sports – A prospective study," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    4. Förster, Bernadette & von der Gracht, Heiko, 2014. "Assessing Delphi panel composition for strategic foresight — A comparison of panels based on company-internal and external participants," Technological Forecasting and Social Change, Elsevier, vol. 84(C), pages 215-229.
    5. Kopyto, Matthias & Lechler, Sabrina & von der Gracht, Heiko A. & Hartmann, Evi, 2020. "Potentials of blockchain technology in supply chain management: Long-term judgments of an international expert panel," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    6. Matsuno, Ken & Kohlbacher, Florian, 2020. "Proactive marketing response to population aging: The roles of capabilities and commitment of firms," Journal of Business Research, Elsevier, vol. 113(C), pages 93-104.
    7. Di Zio, Simone & Bolzan, Mario & Marozzi, Marco, 2021. "Classification of Delphi outputs through robust ranking and fuzzy clustering for Delphi-based scenarios," Technological Forecasting and Social Change, Elsevier, vol. 173(C).
    8. Fritschy, Carolin & Spinler, Stefan, 2019. "The impact of autonomous trucks on business models in the automotive and logistics industry–a Delphi-based scenario study," Technological Forecasting and Social Change, Elsevier, vol. 148(C).
    9. Kanchanabha, Bhawini & Badir, Yuosre F., 2021. "Top management Team's cognitive diversity and the Firm's ambidextrous innovation capability: The mediating role of ambivalent interpretation," Technology in Society, Elsevier, vol. 64(C).
    10. Geissler, Dominik & Beiderbeck, Daniel & Schmidt, Sascha L. & Schreyer, Dominik, 2024. "Emerging technologies and shifting consumer motives: Projecting the future of the top-tier sports media product," Technological Forecasting and Social Change, Elsevier, vol. 203(C).
    11. Spickermann, Alexander & Zimmermann, Martin & von der Gracht, Heiko A., 2014. "Surface- and deep-level diversity in panel selection — Exploring diversity effects on response behaviour in foresight," Technological Forecasting and Social Change, Elsevier, vol. 85(C), pages 105-120.
    12. Marcin Kozak & Olesia Iefremova, 2014. "Implementation Of The Delphi Technique In Finance," "e-Finanse", University of Information Technology and Management, Institute of Financial Research and Analysis, vol. 10(4), pages 36-45, May.
    13. Kent D. Miller & Shu-Jou Lin, 2015. "Analogical reasoning for diagnosing strategic issues in dynamic and complex environments," Strategic Management Journal, Wiley Blackwell, vol. 36(13), pages 2000-2020, December.
    14. Merfeld, Katrin & Wilhelms, Mark-Philipp & Henkel, Sven & Kreutzer, Karin, 2019. "Carsharing with shared autonomous vehicles: Uncovering drivers, barriers and future developments – A four-stage Delphi study," Technological Forecasting and Social Change, Elsevier, vol. 144(C), pages 66-81.
    15. Beverly B. Tyler & Turanay Caner, 2016. "New product introductions below aspirations, slack and R&D alliances: A behavioral perspective," Strategic Management Journal, Wiley Blackwell, vol. 37(5), pages 896-910, May.
    16. Yuan, Wenlong & Bao, Yongjian & Olson, Bradley J., 2017. "CEOs’ ambivalent interpretations, organizational market capabilities, and corporate entrepreneurship as responses to strategic issues," Journal of World Business, Elsevier, vol. 52(2), pages 312-326.
    17. Blake E. Ashforth & Kristie M. Rogers & Michael G. Pratt & Camille Pradies, 2014. "Ambivalence in Organizations: A Multilevel Approach," Organization Science, INFORMS, vol. 25(5), pages 1453-1478, October.
    18. Prommer, Lisa & Tiberius, Victor & Kraus, Sascha, 2020. "Exploring the future of startup leadership development," Journal of Business Venturing Insights, Elsevier, vol. 14(C).
    19. Alyami, Saleh. H. & Rezgui, Yacine & Kwan, Alan, 2013. "Developing sustainable building assessment scheme for Saudi Arabia: Delphi consultation approach," Renewable and Sustainable Energy Reviews, Elsevier, vol. 27(C), pages 43-54.
    20. Elizabeth Gibson & Tugrul Daim & Edwin Garces & Marina Dabic, 2018. "Technology Foresight: A Bibliometric Analysis to Identify Leading and Emerging Methods," Foresight and STI Governance (Foresight-Russia till No. 3/2015), National Research University Higher School of Economics, vol. 12(1), pages 6-24.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:fufsci:v:3:y:2021:i:1:n:e56. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)2573-5152 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.