IDEAS home Printed from https://ideas.repec.org/a/eee/jobhdp/v162y2021icp81-94.html
   My bibliography  Save this article

A concrete example of construct construction in natural language

Author

Listed:
  • Yeomans, Michael

Abstract

Concreteness is central to theories of learning in psychology and organizational behavior. However, the literature provides many competing measures of concreteness in natural language. Indeed, researcher degrees of freedom are often large in text analysis. Here, we use concreteness as an example case for how language measures can be systematically evaluated across many studies. We compare many existing measures across datasets from several domains, including written advice, and plan-making (total N = 9,780). We find that many previous measures have surprisingly little measurement validity in our domains of interest. We also show that domain-specific machine learning models consistently outperform domain-general measures. Text analysis is increasingly common, and our work demonstrates how reproducibility and open data can improve measurement validity for high-dimensional data. We conclude with robust guidelines for measuring concreteness, along with a corresponding R package, doc2concrete, as an open-source toolkit for future research.

Suggested Citation

  • Yeomans, Michael, 2021. "A concrete example of construct construction in natural language," Organizational Behavior and Human Decision Processes, Elsevier, vol. 162(C), pages 81-94.
    • Handle: RePEc:eee:jobhdp:v:162:y:2021:i:c:p:81-94
    DOI: 10.1016/j.obhdp.2020.10.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0749597820303940
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.obhdp.2020.10.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Eggleston, Casey M. & Wilson, Timothy D. & Lee, Minha & Gilbert, Daniel T., 2015. "Predicting what we will like: Asking a stranger can be as good as asking a friend," Organizational Behavior and Human Decision Processes, Elsevier, vol. 128(C), pages 1-10.
    2. Kraft, Matthew A. & Rogers, Todd, 2015. "The underutilized potential of teacher-to-parent communication: Evidence from a field experiment," Economics of Education Review, Elsevier, vol. 47(C), pages 49-63.
    3. Parhankangas, Annaleena & Renko, Maija, 2017. "Linguistic style and crowdfunding success among social and commercial entrepreneurs," Journal of Business Venturing, Elsevier, vol. 32(2), pages 215-236.
    4. Katherine L. Milkman & John Beshears & James J. Choi & David Laibson & Brigitte C. Madrian, 2011. "Using Implementation Intentions Prompts to Enhance Influenza Vaccination Rates," NBER Working Papers 17183, National Bureau of Economic Research, Inc.
    5. Lingling Pan & Gerry McNamara & Jennifer J. Lee & Jerayr (John) Haleblian & Cynthia E. Devers, 2018. "Give it to us straight (most of the time): Top managers’ use of concrete language and its effect on investor reactions," Strategic Management Journal, Wiley Blackwell, vol. 39(8), pages 2204-2225, August.
    6. Claudia Townsend & Wendy Liu, 2012. "Is Planning Good for You? The Differential Impact of Planning on Self-Regulation," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 39(4), pages 688-703.
    7. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    8. Denny, Matthew J. & Spirling, Arthur, 2018. "Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It," Political Analysis, Cambridge University Press, vol. 26(2), pages 168-189, April.
    9. Friedman, Jerome H. & Hastie, Trevor & Tibshirani, Rob, 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 33(i01).
    10. Gaby A. C. Schellekens & Peeter W. J. Verlegh & Ale Smidts, 2010. "Language Abstraction in Word of Mouth," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 37(2), pages 207-223, August.
    11. Sendhil Mullainathan & Jann Spiess, 2017. "Machine Learning: An Applied Econometric Approach," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 87-106, Spring.
    12. Allison, Thomas H. & McKenny, Aaron F. & Short, Jeremy C., 2013. "The effect of entrepreneurial rhetoric on microlending investment: An examination of the warm-glow effect," Journal of Business Venturing, Elsevier, vol. 28(6), pages 690-707.
    13. Bonaccio, Silvia & Dalal, Reeshad S., 2006. "Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 127-151, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Moore, Don A. & Thau, Stefan & Zhong, Chenbo & Gino, Francesca, 2022. "Open Science at OBHDP," Organizational Behavior and Human Decision Processes, Elsevier, vol. 168(C).
    2. Horbach, Serge P.J.M. & Schneider, Jesper W. & Sainte-Marie, Maxime, 2022. "Ungendered writing: Writing styles are unlikely to account for gender differences in funding rates in the natural and technical sciences," Journal of Informetrics, Elsevier, vol. 16(4).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andres Algaba & David Ardia & Keven Bluteau & Samuel Borms & Kris Boudt, 2020. "Econometrics Meets Sentiment: An Overview Of Methodology And Applications," Journal of Economic Surveys, Wiley Blackwell, vol. 34(3), pages 512-547, July.
    2. Lundberg, Ian & Brand, Jennie E. & Jeon, Nanum, 2022. "Researcher reasoning meets computational capacity: Machine learning for social science," SocArXiv s5zc8, Center for Open Science.
    3. Bryan T. Kelly & Asaf Manela & Alan Moreira, 2019. "Text Selection," NBER Working Papers 26517, National Bureau of Economic Research, Inc.
    4. Ratzanyel Rincón, 2023. "Quarterly multidimensional poverty estimates in Mexico using machine learning algorithms/Estimaciones trimestrales de pobreza multidimensional en México mediante algoritmos de aprendizaje de máquina," Estudios Económicos, El Colegio de México, Centro de Estudios Económicos, vol. 38(1), pages 3-68.
    5. Mohamed M. Mostafa, 2023. "A one-hundred-year structural topic modeling analysis of the knowledge structure of international management research," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3905-3935, August.
    6. Berns, John P. & Jia, Yankun & Gondo, Maria, 2022. "Crowdfunding success in sustainability-oriented projects: An exploratory examination of the crowdfunding of 3D printers," Technology in Society, Elsevier, vol. 71(C).
    7. Giovanni Di Franco & Michele Santurro, 2021. "Machine learning, artificial neural networks and social research," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(3), pages 1007-1025, June.
    8. Andrei Dubovik & Adam Elbourne & Bram Hendriks & Mark Kattenberg, 2022. "Forecasting World Trade Using Big Data and Machine Learning Techniques," CPB Discussion Paper 441, CPB Netherlands Bureau for Economic Policy Analysis.
    9. Purwoko Haryadi Santoso & Edi Istiyono & Haryanto & Wahyu Hidayatulloh, 2022. "Thematic Analysis of Indonesian Physics Education Research Literature Using Machine Learning," Data, MDPI, vol. 7(11), pages 1-41, October.
    10. Camilla Salvatore & Silvia Biffignandi & Annamaria Bianchi, 2022. "Corporate Social Responsibility Activities Through Twitter: From Topic Model Analysis to Indexes Measuring Communication Characteristics," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 164(3), pages 1217-1248, December.
    11. Jason Anastasopoulos & George J. Borjas & Gavin G. Cook & Michael Lachanski, 2018. "Job Vacancies, the Beveridge Curve, and Supply Shocks: The Frequency and Content of Help-Wanted Ads in Pre- and Post-Mariel Miami," NBER Working Papers 24580, National Bureau of Economic Research, Inc.
    12. Hadar Gafni & Marek Hudon & Anaïs Périlleux, 2021. "Business or Basic Needs? The Impact of Loan Purpose on Social Crowdfunding Platforms," Journal of Business Ethics, Springer, vol. 173(4), pages 777-793, November.
    13. Andrew B. Martinez, 2020. "Forecast Accuracy Matters for Hurricane Damage," Econometrics, MDPI, vol. 8(2), pages 1-24, May.
    14. Michael J. Weir & Thomas W. Sproul, 2019. "Identifying Drivers of Genetically Modified Seafood Demand: Evidence from a Choice Experiment," Sustainability, MDPI, vol. 11(14), pages 1-21, July.
    15. Laura Toschi & Elisa Ughetto & Andrea Fronzetti Colladon, 2023. "The identity of social impact venture capitalists: exploring social linguistic positioning and linguistic distinctiveness through text mining," Small Business Economics, Springer, vol. 60(3), pages 1249-1280, March.
    16. Anglin, Aaron H. & Wolfe, Marcus T. & Short, Jeremy C. & McKenny, Aaron F. & Pidduck, Robert J., 2018. "Narcissistic rhetoric and crowdfunding performance: A social role theory perspective," Journal of Business Venturing, Elsevier, vol. 33(6), pages 780-812.
    17. Maria Figueroa-Armijos & John P. Berns, 2022. "Vulnerable Populations and Individual Social Responsibility in Prosocial Crowdfunding: Does the Framing Matter for Female and Rural Entrepreneurs?," Journal of Business Ethics, Springer, vol. 177(2), pages 377-394, May.
    18. Achim Ahrens & Christian B. Hansen & Mark E. Schaffer, 2020. "lassopack: Model selection and prediction with regularized regression in Stata," Stata Journal, StataCorp LP, vol. 20(1), pages 176-235, March.
    19. Oo, Pyayt P. & Jiang, Lin & Sahaym, Arvin & Parhankangas, Annaleena & Chan, Richard, 2023. "Actions in words: How entrepreneurs use diversified and changing speech acts to achieve funding success," Journal of Business Venturing, Elsevier, vol. 38(2).
    20. Holger Breinlich & Valentina Corradi & Nadia Rocha & Michele Ruta & Joao M.C. Santos Silva & Tom Zylkin, 2021. "Machine Learning in International Trade Research ?- Evaluating the Impact of Trade Agreements," School of Economics Discussion Papers 0521, School of Economics, University of Surrey.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jobhdp:v:162:y:2021:i:c:p:81-94. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/obhdp .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.