Quantifying Deregulation and its Economic Effects: A Large Language Model Approach

We construct a news-based index of deregulation for the United States from 1960 through 2025 using AI to semantically classify newspaper articles. We distinguish articles discussing deregulation from those discussing increased regulation, assigning intensity scores that reflect both the centrality of deregulatory content and whether articles discuss advocacy, proposals, or enacted measures. Human validation confirms strong agreement between AI and human classifications. The deregulation index captures major reform episodes including transportation and telecommunications liberalization in the 1970s--1980s, financial deregulation in the 1980s-1990s, and recent deregulatory activity. We decompose the index by sector, type of deregulation, and policy stage. We validate the news-based index against a parallel index constructed using Federal Register documents: the news-based index leads the Federal Register index by nearly one year, consistent with media coverage reflecting policy intentions before formal implementation. Unlike measures based on detailed statutory coding or Federal Register counts that weigh all rules equally, our approach covers the entire economy and weighs naturally by newsworthiness, capturing regulatory shifts before they materialize in law. Positive shocks to deregulation boost investment, productivity, stock prices, profits, and GDP. Industry-specific deregulation shocks boost industry-level stock returns, consistent with our finding that deregulation involves measures that may impact incumbent profitability and operational efficiency more than competitive entry.