IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2604.02403.html

Measuring What Cannot Be Surveyed: LLMs as Instruments for Latent Cognitive Variables in Labor Economics

Author

Listed:
  • Cristian Espinal Maya

Abstract

This paper establishes the theoretical and practical foundations for using Large Language Models (LLMs) as measurement instruments for latent economic variables -- specifically variables that describe the cognitive content of occupational tasks at a level of granularity not achievable with existing survey instruments. I formalize four conditions under which LLM-generated scores constitute valid instruments: semantic exogeneity, construct relevance, monotonicity, and model invariance. I then apply this framework to the Augmented Human Capital Index (AHC_o), constructed from 18,796 O*NET task statements scored by Claude Haiku 4.5, and validated against six existing AI exposure indices. The index shows strong convergent validity (r = 0.85 with Eloundou GPT-gamma, r = 0.79 with Felten AIOE) and discriminant validity. Principal component analysis confirms that AI-related occupational measures span two distinct dimensions -- augmentation and substitution. Inter-rater reliability across two LLM models (n = 3,666 paired scores) yields Pearson r = 0.76 and Krippendorff's alpha = 0.71. Prompt sensitivity analysis across four alternative framings shows that task-level rankings are robust. Obviously Related Instrumental Variables (ORIV) estimation recovers coefficients 25% larger than OLS, consistent with classical measurement error attenuation. The methodology generalizes beyond labor economics to any domain where semantic content must be quantified at scale.

Suggested Citation

  • Cristian Espinal Maya, 2026. "Measuring What Cannot Be Surveyed: LLMs as Instruments for Latent Cognitive Variables in Labor Economics," Papers 2604.02403, arXiv.org.
  • Handle: RePEc:arx:papers:2604.02403
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2604.02403
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Edward Felten & Manav Raj & Robert Seamans, 2021. "Occupational, industry, and geographic exposure to artificial intelligence: A novel dataset and its potential uses," Strategic Management Journal, Wiley Blackwell, vol. 42(12), pages 2195-2217, December.
    2. Cristian Espinal Maya, 2026. "Augmented Human Capital: A Unified Theory and LLM-Based Measurement Framework for Cognitive Factor Decomposition in AI-Augmented Economies," Papers 2604.01066, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zara Contractor & Germ'an Reyes, 2025. "Generative AI in Higher Education: Evidence from an Elite College," Papers 2508.00717, arXiv.org, revised Apr 2026.
    2. Christos Makridis & Christos A. Makridis, 2025. "The Labor Market Effect of Generative Artificial Intelligence on Artists," CESifo Working Paper Series 12368, CESifo.
    3. Czarnitzki, Dirk & Fernández, Gastón P. & Rammer, Christian, 2023. "Artificial intelligence and firm-level productivity," Journal of Economic Behavior & Organization, Elsevier, vol. 211(C), pages 188-205.
    4. Federico Riccio & Jacopo Staccioli & Maria Enrica Virgillito, 2025. "European regional employment and exposure to labour-saving technical change: results from a direct text similarity measure," LEM Papers Series 2025/19, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    5. Enrico Maria Fenoaltea & Dario Mazzilli & Aurelio Patelli & Angelica Sbardella & Andrea Tacchella & Andrea Zaccaria & Marco Trombetti & Luciano Pietronero, 2024. "Follow the money: a startup-based measure of AI exposure across occupations, industries and regions," Papers 2412.04924, arXiv.org, revised Dec 2024.
    6. Benedetta Montanaro & Annalisa Croce & Elisa Ughetto, 2024. "Venture capital investments in artificial intelligence," Journal of Evolutionary Economics, Springer, vol. 34(1), pages 1-28, January.
    7. Cameron D. Miller & Richard D. Wang, 2024. "Product digitization and differentiation strategy change: Evidence from the book publishing industry," Strategic Management Journal, Wiley Blackwell, vol. 45(7), pages 1241-1272, July.
    8. Caselli, Mauro & Fracasso, Andrea & Scicchitano, Sergio & Traverso, Silvio & Tundis, Enrico, 2025. "What workers and robots do: An activity-based analysis of the impact of robotization on changes in local employment," Research Policy, Elsevier, vol. 54(1).
    9. Yusuke Aoki & Joon Suk Park & Yuya Takada & Koji Takahashi, 2025. "Expecting Job Replacement by GenAI: Effects on Workers' Economic Outlook and Behavior," IMES Discussion Paper Series 25-E-04, Institute for Monetary and Economic Studies, Bank of Japan.
    10. Milan Miric & Nan Jia & Kenneth G. Huang, 2023. "Using supervised machine learning for large‐scale classification in management research: The case for identifying artificial intelligence patents," Strategic Management Journal, Wiley Blackwell, vol. 44(2), pages 491-519, February.
    11. Jin, Cheng-Xiao & Yu, Jia-Qi, 2025. "Does the popularization of artificial intelligence increase the risk of corporate digital responsibility?," Finance Research Letters, Elsevier, vol. 80(C).
    12. Standaert, Thomas & Andries, Petra, 2026. "Overcoming difficulties in knowledge transfer: Harnessing the power of AI to drive process innovation," Technovation, Elsevier, vol. 149(C).
    13. Flavio Calvino & Luca Fontanelli, 2026. "Decoding AI: an early look at how French firms use AI," Eurasian Business Review, Springer;Eurasia Business and Economics Society, vol. 16(1), pages 51-93, March.
    14. Yusuke Aoki & Joon Suk Park & Yuya Takada & Koji Takahashi, 2025. "Expecting job replacement by GenAI: effects on workers' economic outlook and behavior," BIS Working Papers 1269, Bank for International Settlements.
    15. Megan Lawrence & Christopher Poliquin, 2023. "The growth of hierarchy in organizations: Managing knowledge scope," Strategic Management Journal, Wiley Blackwell, vol. 44(13), pages 3155-3184, December.
    16. Konstantinos Pouliakas & Giulia Santangelo & Paul Dupire, 2025. "Are artificial intelligence skills a reward or a gamble? Deconstructing the AI wage premium in Europe," Eurasian Business Review, Springer;Eurasia Business and Economics Society, vol. 15(4), pages 1091-1128, December.
    17. Liu, Qing & Lu, Ruosi & Teng Sun, Stephen & Zhang, Meng, 2024. "Unintended workplace safety consequences of minimum wages," Journal of Public Economics, Elsevier, vol. 239(C).
    18. Mammadov Huseyn & Africa Ruiz-Gandara & Luis Gonzalez-Abril & Isidoro Romero, 2024. "Adoption of Artificial Intelligence in Small and Medium-Sized Enterprises in Spain: The Role of Competences and Skills," The AMFITEATRU ECONOMIC journal, Academy of Economic Studies - Bucharest, Romania, vol. 26(67), pages 848-848, August.
    19. Guarascio, Dario & Reljic, Jelena & Stöllinger, Roman, 2025. "Diverging paths: AI exposure and employment across European regions," Structural Change and Economic Dynamics, Elsevier, vol. 73(C), pages 11-24.
    20. Chiarello, Filippo & Fantoni, Gualtiero & Hogarth, Terence & Giordano, Vito & Baltina, Liga & Spada, Irene, 2021. "Towards ESCO 4.0 – Is the European classification of skills in line with Industry 4.0? A text mining approach," Technological Forecasting and Social Change, Elsevier, vol. 173(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2604.02403. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.