IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/129904.html

Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not

Author

Listed:
  • Rai, Sudhanshu

Abstract

A growing empirical literature uses pre-built "AI occupational exposure" scores (most prominently the AI Occupational Exposure (AIOE) index of Felten, Raj, and Seamans (2021) and the GPT-4 task-exposure measure of Eloundou et al. (2024)) as occupation-level treatments or predictors for AI's labor-market effects. We show that two of the most-cited scores, AIOE and Eloundou's GPT-4 measure, substantially re-label cognitive task content rather than capturing AI-specific exposure, a construct-validity problem that does not extend to a third, differently-built score (Webb 2020, patent-based). This note horse-races these three measures against transparent cognitive/manual task-content indices and the established Autor–Dorn Routine Task Intensity (RTI) measure. Across 773 occupations: (i) the ten-plus AI "applications" underlying AIOE collapse to a single factor (first principal component ≈ 88%); (ii) AIOE and Eloundou each correlate strongly with a cognitive-ability index (+0.85 / +0.70) and negatively with a manual-ability index (−0.91 / −0.83), correlate 0.86 with each other, but only moderately with RTI (−0.33 / −0.30): the confound is specifically cognitive ability level, not the classic routine-task polarization axis; (iii) each score's positive wage association reverses sign controlling for cognitive content but is barely affected by controlling for RTI; and (iv) the much-cited pre-ChatGPT "AI foresight" wage-divergence pattern collapses to near-zero under the cognitive control (not under the RTI control). Critically, this collapse is not universal: Webb's patent-text-overlap score is essentially uncorrelated with AIOE (r=0.03) and Eloundou (r=−0.03), only weakly related to cognitive content (r=0.13), and its modest wage associations do not reverse under cognitive control. The cognitive-content collapse is a signature of how a score is built: subjective crowd-relatedness ratings (AIOE) or LLM/human task judgments (Eloundou), not an inherent property of occupational AI-exposure measurement. Studies using relatedness- or judgment-based exposure scores should control for cognitive content and re-interpret accordingly; a patent-based measure is not shown here to have the same problem, though its own construct validity is untested.

Suggested Citation

  • Rai, Sudhanshu, 2026. "Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not," MPRA Paper 129904, University Library of Munich, Germany.
  • Handle: RePEc:pra:mprapa:129904
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/129904/1/MPRA_paper_129904.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • J24 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Human Capital; Skills; Occupational Choice; Labor Productivity
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:129904. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.