Content-based Metric on Monetary Policy Uncertainty by Using Large Language Models

Policy uncertainty has the potential to reduce policy effectiveness. Existing studies have measured policy uncertainty by tracking the frequency of specific keywords in newspaper articles. However, this keyword-based approach fails to account for the context of the articles and differentiate the types of uncertainty that such contexts indicate. This study introduces a new method of measuring different types of policy uncertainty in news content which utilizes large language models (LLMs). Specifically, we differentiate policy uncertainty into forward-looking and backward-looking uncertainty, or in other words, uncertainty regarding future policy direction and uncertainty about the effectiveness of the current policy. We fine-tune the LLMs to identify each type of uncertainty expressed in newspaper articles based on their context, even in the absence of specific keywords indicating uncertainty. By applying this method, we measure Japan’s monetary policy uncertainty (MPU) from 2015 to 2016. To reflect the unprecedented monetary policy conditions during this period when the unconventional policies were taken, we further classify MPU by layers of policy changes: changes in specific market operations and changes in the broader policy framework. The experimental results show that our approach successfully captures the dynamics of MPU, particularly for forward-looking uncertainty, which is not fully captured by the existing approach. Forward- and backward-looking uncertainty indices exhibit distinct movements depending on the conditions under which changes in the policy framework occur. This suggests that perceived uncertainty regarding monetary policy would be state-dependent, varying with the prevailing social environment.