IDEAS home Printed from https://ideas.repec.org/a/wly/jforec/v44y2025i4p1383-1402.html
   My bibliography  Save this article

Forecasting the Volatility of US Oil and Gas Firms With Machine Learning

Author

Listed:
  • Juan D. Díaz
  • Erwin Hansen
  • Gabriel Cabrera

Abstract

Forecasting the realized volatility of oil and gas firms is of interest to investors and practitioners trading on the energy spot and derivative markets. In this paper, we assess whether several machine learning (ML) techniques can offer superior forecasts compared to HAR models for predicting realized volatility at the firm level. Moreover, we investigate whether economically motivated variables and technical indicators contain valuable information for forecasting firm volatility beyond those contained in various volatility factors previously identified in the literature. Our results demonstrate that certain ML techniques provide superior forecasting accuracy compared to the benchmark model. Additionally, we identify variables such as the 1‐month treasury bill and the aggregate VIX index as significant drivers of realized firm volatility in the oil and gas industry.

Suggested Citation

  • Juan D. Díaz & Erwin Hansen & Gabriel Cabrera, 2025. "Forecasting the Volatility of US Oil and Gas Firms With Machine Learning," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 44(4), pages 1383-1402, July.
    • Handle: RePEc:wly:jforec:v:44:y:2025:i:4:p:1383-1402
    DOI: 10.1002/for.3245
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/for.3245
    Download Restriction: no
    File URL: https://libkey.io/10.1002/for.3245?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Andrea Bucci, 2020. "Realized Volatility Forecasting with Neural Networks," Journal of Financial Econometrics, Oxford University Press, vol. 18(3), pages 502-531.
    2. Lubos Pástor & Pietro Veronesi, 2012. "Uncertainty about Government Policy and Stock Prices," Journal of Finance, American Finance Association, vol. 67(4), pages 1219-1264, August.
    3. Zhifeng Dai & Tingyu Li & Mi Yang, 2022. "Forecasting stock return volatility: The role of shrinkage approaches in a data‐rich environment," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 980-996, August.
    4. Charlotte Christiansen & Maik Schmeling & Andreas Schrimpf, 2012. "A comprehensive look at financial volatility prediction by economic variables," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 956-977, September.
    5. Christopher J. Neely & David E. Rapach & Jun Tu & Guofu Zhou, 2014. "Forecasting the Equity Risk Premium: The Role of Technical Indicators," Management Science, INFORMS, vol. 60(7), pages 1772-1791, July.
    6. Kim Christensen & Mathias Siggaard & Bezirgen Veliyev, 2023. "A Machine Learning Approach to Volatility Forecasting," Journal of Financial Econometrics, Oxford University Press, vol. 21(5), pages 1680-1727.
    7. Ruipeng Liu & Rangan Gupta, 2022. "Investors’ Uncertainty and Forecasting Stock Market Volatility," Journal of Behavioral Finance, Taylor & Francis Journals, vol. 23(3), pages 327-337, July.
    8. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    9. Mittnik, Stefan & Robinzonov, Nikolay & Spindler, Martin, 2015. "Stock market volatility: Identifying major drivers and the nature of their impact," Journal of Banking & Finance, Elsevier, vol. 58(C), pages 1-14.
    10. DeMiguel, Victor & Gil-Bazo, Javier & Nogales, Francisco J. & Santos, André A.P., 2023. "Machine learning and fund characteristics help to select mutual funds with positive alpha," Journal of Financial Economics, Elsevier, vol. 150(3).
    11. Liu, Li & Pan, Zhiyuan, 2020. "Forecasting stock market volatility: The role of technical variables," Economic Modelling, Elsevier, vol. 84(C), pages 55-65.
    12. Nonejad, Nima, 2017. "Forecasting aggregate stock market volatility using financial and macroeconomic predictors: Which models forecast best, when and why?," Journal of Empirical Finance, Elsevier, vol. 42(C), pages 131-154.
    13. Patton, Andrew J., 2011. "Volatility forecast comparison using imperfect volatility proxies," Journal of Econometrics, Elsevier, vol. 160(1), pages 246-256, January.
    14. Ma, Feng & Liu, Jing & Wahab, M.I.M. & Zhang, Yaojie, 2018. "Forecasting the aggregate oil price volatility in a data-rich environment," Economic Modelling, Elsevier, vol. 72(C), pages 320-332.
    15. Li Liu & Feng Ma & Qing Zeng & Yaojie Zhang, 2020. "Forecasting the aggregate stock market volatility in a data-rich world," Applied Economics, Taylor & Francis Journals, vol. 52(32), pages 3448-3463, June.
    16. Zhang, Yaojie & Wei, Yu & Zhang, Yi & Jin, Daxiang, 2019. "Forecasting oil price volatility: Forecast combination versus shrinkage method," Energy Economics, Elsevier, vol. 80(C), pages 423-433.
    17. Antonakakis, Nikolaos & Cunado, Juncal & Filis, George & Gabauer, David & Perez de Gracia, Fernando, 2018. "Oil volatility, oil and gas firms and portfolio diversification," Energy Economics, Elsevier, vol. 70(C), pages 499-515.
    18. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    19. Zhang, Yaojie & Ma, Feng & Wang, Yudong, 2019. "Forecasting crude oil prices with a large set of predictors: Can LASSO select powerful predictors?," Journal of Empirical Finance, Elsevier, vol. 54(C), pages 97-117.
    20. Hongwei Zhang & Qiang He & Ben Jacobsen & Fuwei Jiang, 2020. "Forecasting stock returns with model uncertainty and parameter instability," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 629-644, August.
    21. Zhu, Haibin & Bai, Lu & He, Lidan & Liu, Zhi, 2023. "Forecasting realized volatility with machine learning: Panel data perspective," Journal of Empirical Finance, Elsevier, vol. 73(C), pages 251-271.
    22. Friedman, Jerome H. & Hastie, Trevor & Tibshirani, Rob, 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 33(i01).
    23. He, Mengxi & Zhang, Yaojie & Wen, Danyan & Wang, Yudong, 2021. "Forecasting crude oil prices: A scaled PCA approach," Energy Economics, Elsevier, vol. 97(C).
    24. Yin, Libo & Yang, Qingyuan, 2016. "Predicting the oil prices: Do technical indicators help?," Energy Economics, Elsevier, vol. 56(C), pages 338-350.
    25. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2024. "Machine-learning stock market volatility: Predictability, drivers, and economic value," International Review of Financial Analysis, Elsevier, vol. 94(C).
    26. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    27. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2024. "Machine-learning stock market volatility: Predictability, drivers, and economic value," International Review of Financial Analysis, Elsevier, vol. 94(C).
    2. Wen, Danyan & He, Mengxi & Wang, Yudong & Zhang, Yaojie, 2024. "Forecasting crude oil market volatility: A comprehensive look at uncertainty variables," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1022-1041.
    3. Zhang, Yaojie & Wahab, M.I.M. & Wang, Yudong, 2023. "Forecasting crude oil market volatility using variable selection and common factor," International Journal of Forecasting, Elsevier, vol. 39(1), pages 486-502.
    4. Gong, Xue & Ye, Xin & Zhang, Weiguo & Zhang, Yue, 2023. "Predicting energy futures high-frequency volatility using technical indicators: The role of interaction," Energy Economics, Elsevier, vol. 119(C).
    5. Li, Xiaodan & Gong, Xue & Ge, Futing & Huang, Jingjing, 2024. "Forecasting stock volatility using pseudo-out-of-sample information," International Review of Economics & Finance, Elsevier, vol. 90(C), pages 123-135.
    6. Zhu, Haibin & Bai, Lu & He, Lidan & Liu, Zhi, 2023. "Forecasting realized volatility with machine learning: Panel data perspective," Journal of Empirical Finance, Elsevier, vol. 73(C), pages 251-271.
    7. He, Mengxi & Wang, Yudong & Zeng, Qing & Zhang, Yaojie, 2023. "Forecasting aggregate stock market volatility with industry volatilities: The role of spillover index," Research in International Business and Finance, Elsevier, vol. 65(C).
    8. He, Mengxi & Wen, Danyan & Xing, Lu & Zhang, Yaojie, 2024. "Industry volatility concentration and the predictability of aggregate stock market volatility," International Review of Economics & Finance, Elsevier, vol. 95(C).
    9. Guo, Yangli & Ma, Feng & Li, Haibo & Lai, Xiaodong, 2022. "Oil price volatility predictability based on global economic conditions," International Review of Financial Analysis, Elsevier, vol. 82(C).
    10. Niu, Zibo & Demirer, Riza & Suleman, Muhammad Tahir & Zhang, Hongwei & Zhu, Xuehong, 2024. "Do industries predict stock market volatility? Evidence from machine learning models," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 90(C).
    11. Niu, Zibo & Wang, Chenlu & Zhang, Hongwei, 2023. "Forecasting stock market volatility with various geopolitical risks categories: New evidence from machine learning models," International Review of Financial Analysis, Elsevier, vol. 89(C).
    12. Zhang, Hongwei & Wang, Wentao & Niu, Zibo, 2024. "Geopolitical risks and crude oil futures volatility: Evidence from machine learning," Resources Policy, Elsevier, vol. 98(C).
    13. Mengxi He & Xianfeng Hao & Yaojie Zhang & Fanyi Meng, 2021. "Forecasting stock return volatility using a robust regression model," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(8), pages 1463-1478, December.
    14. Zhifeng Dai & Tingyu Li & Mi Yang, 2022. "Forecasting stock return volatility: The role of shrinkage approaches in a data‐rich environment," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 980-996, August.
    15. Lu, Xinjie & Ma, Feng & Xu, Jin & Zhang, Zehui, 2022. "Oil futures volatility predictability: New evidence based on machine learning models11All the authors contribute to the paper equally," International Review of Financial Analysis, Elsevier, vol. 83(C).
    16. Zeng, Qing & Lu, Xinjie & Xu, Jin & Lin, Yu, 2024. "Macro-Driven Stock Market Volatility Prediction: Insights from a New Hybrid Machine Learning Approach," International Review of Financial Analysis, Elsevier, vol. 96(PB).
    17. Xue Gong & Weiguo Zhang & Yuan Zhao & Xin Ye, 2023. "Forecasting stock volatility with a large set of predictors: A new forecast combination method," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1622-1647, November.
    18. Xing, Li-Min & Zhang, Yue-Jun, 2022. "Forecasting crude oil prices with shrinkage methods: Can nonconvex penalty and Huber loss help?," Energy Economics, Elsevier, vol. 110(C).
    19. Zhang, Hongwei & Zhao, Xinyi & Gao, Wang & Niu, Zibo, 2023. "The role of higher moments in predicting China's oil futures volatility: Evidence from machine learning models," Journal of Commodity Markets, Elsevier, vol. 32(C).
    20. Danyan Wen & Mengxi He & Yaojie Zhang & Yudong Wang, 2022. "Forecasting realized volatility of Chinese stock market: A simple but efficient truncated approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(2), pages 230-251, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:jforec:v:44:y:2025:i:4:p:1383-1402. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.