IDEAS home Printed from https://ideas.repec.org/p/osf/socarx/47gfn_v1.html

Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not

Author

Listed:
  • Rai, Sudhanshu

    (None)

Abstract

A growing empirical literature uses pre-built "AI occupational exposure" scores (most prominently the AI Occupational Exposure (AIOE) index of Felten, Raj, and Seamans (2021) and the GPT-4 task-exposure measure of Eloundou et al. (2024)) as occupation-level treatments or predictors for AI's labor-market effects. We show that two of the most-cited scores, AIOE and Eloundou's GPT-4 measure, substantially re-label cognitive task content rather than capturing AI-specific exposure, a construct-validity problem that does not extend to a third, differently-built score (Webb 2020, patent-based). This note horse-races these three measures against transparent cognitive/manual task-content indices and the established Autor–Dorn Routine Task Intensity (RTI) measure. Across 773 occupations: (i) the ten-plus AI "applications" underlying AIOE collapse to a single factor (first principal component ≈ 88%); (ii) AIOE and Eloundou each correlate strongly with a cognitive-ability index (+0.85 / +0.70) and negatively with a manual-ability index (−0.91 / −0.83), correlate 0.86 with each other, but only moderately with RTI (−0.33 / −0.30): the confound is specifically cognitive ability level, not the classic routine-task polarization axis; (iii) each score's positive wage association reverses sign controlling for cognitive content but is barely affected by controlling for RTI; and (iv) the much-cited pre-ChatGPT "AI foresight" wage-divergence pattern collapses to near-zero under the cognitive control (not under the RTI control). Critically, this collapse is not universal: Webb's patent-text-overlap score is essentially uncorrelated with AIOE (r=0.03) and Eloundou (r=−0.03), only weakly related to cognitive content (r=0.13), and its modest wage associations do not reverse under cognitive control. The cognitive-content collapse is a signature of how a score is built: subjective crowd-relatedness ratings (AIOE) or LLM/human task judgments (Eloundou), not an inherent property of occupational AI-exposure measurement. Studies using relatedness- or judgment-based exposure scores should control for cognitive content and re-interpret accordingly; a patent-based measure is not shown here to have the same problem, though its own construct validity is untested.

Suggested Citation

  • Rai, Sudhanshu, 2026. "Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not," SocArXiv 47gfn_v1, Center for Open Science.
  • Handle: RePEc:osf:socarx:47gfn_v1
    DOI: 10.31235/osf.io/47gfn_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/6aa76ece907023bbdf10c47c/
    Download Restriction: no

    File URL: https://libkey.io/10.31235/osf.io/47gfn_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:socarx:47gfn_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://socarxiv.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.