IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/129904.html

Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not

Author

Listed:
  • Rai, Sudhanshu

Abstract

A growing empirical literature uses pre-built "AI occupational exposure" scores (most prominently the AI Occupational Exposure (AIOE) index of Felten, Raj, and Seamans (2021) and the GPT-4 task-exposure measure of Eloundou et al. (2024)) as occupation-level treatments or predictors for AI's labor-market effects. We show that two of the most-cited scores, AIOE and Eloundou's GPT-4 measure, substantially re-label cognitive task content rather than capturing AI-specific exposure, a construct-validity problem that does not extend to a third, differently-built score (Webb 2020, patent-based). This note horse-races these three measures against transparent cognitive/manual task-content indices and the established Autor–Dorn Routine Task Intensity (RTI) measure. Across 773 occupations: (i) the ten-plus AI "applications" underlying AIOE collapse to a single factor (first principal component ≈ 88%); (ii) AIOE and Eloundou each correlate strongly with a cognitive-ability index (+0.85 / +0.70) and negatively with a manual-ability index (−0.91 / −0.83), correlate 0.86 with each other, but only moderately with RTI (−0.33 / −0.30): the confound is specifically cognitive ability level, not the classic routine-task polarization axis; (iii) each score's positive wage association reverses sign controlling for cognitive content but is barely affected by controlling for RTI; and (iv) the much-cited pre-ChatGPT "AI foresight" wage-divergence pattern collapses to near-zero under the cognitive control (not under the RTI control). Critically, this collapse is not universal: Webb's patent-text-overlap score is essentially uncorrelated with AIOE (r=0.03) and Eloundou (r=−0.03), only weakly related to cognitive content (r=0.13), and its modest wage associations do not reverse under cognitive control. The cognitive-content collapse is a signature of how a score is built: subjective crowd-relatedness ratings (AIOE) or LLM/human task judgments (Eloundou), not an inherent property of occupational AI-exposure measurement. Studies using relatedness- or judgment-based exposure scores should control for cognitive content and re-interpret accordingly; a patent-based measure is not shown here to have the same problem, though its own construct validity is untested.

Suggested Citation

  • Rai, Sudhanshu, 2026. "Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not," MPRA Paper 129904, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:129904
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/129904/1/MPRA_paper_129904.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Edward Felten & Manav Raj & Robert Seamans, 2021. "Occupational, industry, and geographic exposure to artificial intelligence: A novel dataset and its potential uses," Strategic Management Journal, Wiley Blackwell, vol. 42(12), pages 2195-2217, December.
    2. Morgan R. Frank & Alireza Javadian Sabet & Lisa Simon & Sarah H. Bana & Renzhe Yu, 2026. "AI-exposed jobs deteriorated before ChatGPT," Papers 2601.02554, arXiv.org.
    3. Campbell Lund & Thomas Euyang & Zanele Munyikwa & Marzieh Fadaee, 2026. "AI Exposure Scores: what they measure, what they miss, and what comes next," Papers 2606.23633, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zara Contractor & Germ'an Reyes, 2025. "Generative AI in Higher Education: Evidence from an Elite College," Papers 2508.00717, arXiv.org, revised Apr 2026.
    2. Czarnitzki, Dirk & Fernández, Gastón P. & Rammer, Christian, 2023. "Artificial intelligence and firm-level productivity," Journal of Economic Behavior & Organization, Elsevier, vol. 211(C), pages 188-205.
    3. Federico Riccio & Jacopo Staccioli & Maria Enrica Virgillito, 2025. "European regional employment and exposure to labour-saving technical change: results from a direct text similarity measure," LEM Papers Series 2025/19, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    4. Enrico Maria Fenoaltea & Dario Mazzilli & Aurelio Patelli & Angelica Sbardella & Andrea Tacchella & Andrea Zaccaria & Marco Trombetti & Luciano Pietronero, 2024. "Follow the money: a startup-based measure of AI exposure across occupations, industries and regions," Papers 2412.04924, arXiv.org, revised Dec 2024.
    5. Benedetta Montanaro & Annalisa Croce & Elisa Ughetto, 2024. "Venture capital investments in artificial intelligence," Journal of Evolutionary Economics, Springer, vol. 34(1), pages 1-28, January.
    6. Cameron D. Miller & Richard D. Wang, 2024. "Product digitization and differentiation strategy change: Evidence from the book publishing industry," Strategic Management Journal, Wiley Blackwell, vol. 45(7), pages 1241-1272, July.
    7. Caselli, Mauro & Fracasso, Andrea & Scicchitano, Sergio & Traverso, Silvio & Tundis, Enrico, 2025. "What workers and robots do: An activity-based analysis of the impact of robotization on changes in local employment," Research Policy, Elsevier, vol. 54(1).
    8. Yusuke Aoki & Joon Suk Park & Yuya Takada & Koji Takahashi, 2025. "Expecting job replacement by GenAI: effects on workers' economic outlook and behavior," BIS Working Papers 1269, Bank for International Settlements.
    9. Kaibalyapati Mishra, 2026. "The Privilege of Exposure: Caste and Generative AI in India's Graduate Labour Market," Papers 2606.13314, arXiv.org, revised Jun 2026.
    10. Milan Miric & Nan Jia & Kenneth G. Huang, 2023. "Using supervised machine learning for large‐scale classification in management research: The case for identifying artificial intelligence patents," Strategic Management Journal, Wiley Blackwell, vol. 44(2), pages 491-519, February.
    11. John Gilbert & Onur A. Koska & Reza Oladi, 2026. "Tariffs, taxes and wages in the age of artificial intelligence," Economic Inquiry, Western Economic Association International, vol. 64(2), pages 395-408, April.
    12. Standaert, Thomas & Andries, Petra, 2026. "Overcoming difficulties in knowledge transfer: Harnessing the power of AI to drive process innovation," Technovation, Elsevier, vol. 149(C).
    13. Soohyoung Lee & Dawoon Jeong & Jeong-Dong Lee, 2026. "Contrasting pathways of automation: routine task substitution vs. AI complementarity," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 13(1), pages 1-11, December.
    14. Megan Lawrence & Christopher Poliquin, 2023. "The growth of hierarchy in organizations: Managing knowledge scope," Strategic Management Journal, Wiley Blackwell, vol. 44(13), pages 3155-3184, December.
    15. Mammadov Huseyn & Africa Ruiz-Gandara & Luis Gonzalez-Abril & Isidoro Romero, 2024. "Adoption of Artificial Intelligence in Small and Medium-Sized Enterprises in Spain: The Role of Competences and Skills," The AMFITEATRU ECONOMIC journal, Academy of Economic Studies - Bucharest, Romania, vol. 26(67), pages 848-848, August.
    16. Guarascio, Dario & Reljic, Jelena & Stöllinger, Roman, 2025. "Diverging paths: AI exposure and employment across European regions," Structural Change and Economic Dynamics, Elsevier, vol. 73(C), pages 11-24.
    17. Chiarello, Filippo & Fantoni, Gualtiero & Hogarth, Terence & Giordano, Vito & Baltina, Liga & Spada, Irene, 2021. "Towards ESCO 4.0 – Is the European classification of skills in line with Industry 4.0? A text mining approach," Technological Forecasting and Social Change, Elsevier, vol. 173(C).
    18. Zara Contractor & Germán Reyes, 2025. "Generative AI in Higher Education: Evidence from an Elite College," CEDLAS, Working Papers 0359, CEDLAS, Universidad Nacional de La Plata.
    19. A. O. Aver’yanov & I. S. Stepus’ & V. A. Gurtov, 2023. "Forecast of Staffing Needs for the Artificial Intelligence Sector in Russia," Studies on Russian Economic Development, Springer, vol. 34(1), pages 86-95, February.
    20. Koch, Michael & Lodefalk, Magnus, 2024. "Artificial Intelligence and Worker Stress: Evidence from Germany," Ratio Working Papers 377, The Ratio Institute.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:129904. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.