IDEAS home Printed from https://ideas.repec.org/a/wly/jforec/v44y2025i4p1383-1402.html
   My bibliography  Save this article

Forecasting the Volatility of US Oil and Gas Firms With Machine Learning

Author

Listed:
  • Juan D. Díaz
  • Erwin Hansen
  • Gabriel Cabrera

Abstract

Forecasting the realized volatility of oil and gas firms is of interest to investors and practitioners trading on the energy spot and derivative markets. In this paper, we assess whether several machine learning (ML) techniques can offer superior forecasts compared to HAR models for predicting realized volatility at the firm level. Moreover, we investigate whether economically motivated variables and technical indicators contain valuable information for forecasting firm volatility beyond those contained in various volatility factors previously identified in the literature. Our results demonstrate that certain ML techniques provide superior forecasting accuracy compared to the benchmark model. Additionally, we identify variables such as the 1‐month treasury bill and the aggregate VIX index as significant drivers of realized firm volatility in the oil and gas industry.

Suggested Citation

  • Juan D. Díaz & Erwin Hansen & Gabriel Cabrera, 2025. "Forecasting the Volatility of US Oil and Gas Firms With Machine Learning," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 44(4), pages 1383-1402, July.
    • Handle: RePEc:wly:jforec:v:44:y:2025:i:4:p:1383-1402
    DOI: 10.1002/for.3245
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/for.3245
    Download Restriction: no
    File URL: https://libkey.io/10.1002/for.3245?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Antonakakis, Nikolaos & Cunado, Juncal & Filis, George & Gabauer, David & Perez de Gracia, Fernando, 2018. "Oil volatility, oil and gas firms and portfolio diversification," Energy Economics, Elsevier, vol. 70(C), pages 499-515.
    2. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    3. Andrea Bucci, 2020. "Realized Volatility Forecasting with Neural Networks," Journal of Financial Econometrics, Oxford University Press, vol. 18(3), pages 502-531.
    4. Zhang, Yaojie & Ma, Feng & Wang, Yudong, 2019. "Forecasting crude oil prices with a large set of predictors: Can LASSO select powerful predictors?," Journal of Empirical Finance, Elsevier, vol. 54(C), pages 97-117.
    5. Lubos Pástor & Pietro Veronesi, 2012. "Uncertainty about Government Policy and Stock Prices," Journal of Finance, American Finance Association, vol. 67(4), pages 1219-1264, August.
    6. Hongwei Zhang & Qiang He & Ben Jacobsen & Fuwei Jiang, 2020. "Forecasting stock returns with model uncertainty and parameter instability," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 629-644, August.
    7. Zhu, Haibin & Bai, Lu & He, Lidan & Liu, Zhi, 2023. "Forecasting realized volatility with machine learning: Panel data perspective," Journal of Empirical Finance, Elsevier, vol. 73(C), pages 251-271.
    8. Friedman, Jerome H. & Hastie, Trevor & Tibshirani, Rob, 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 33(i01).
    9. He, Mengxi & Zhang, Yaojie & Wen, Danyan & Wang, Yudong, 2021. "Forecasting crude oil prices: A scaled PCA approach," Energy Economics, Elsevier, vol. 97(C).
    10. Charlotte Christiansen & Maik Schmeling & Andreas Schrimpf, 2012. "A comprehensive look at financial volatility prediction by economic variables," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 956-977, September.
    11. Ruipeng Liu & Rangan Gupta, 2022. "Investors’ Uncertainty and Forecasting Stock Market Volatility," Journal of Behavioral Finance, Taylor & Francis Journals, vol. 23(3), pages 327-337, July.
    12. Zhifeng Dai & Tingyu Li & Mi Yang, 2022. "Forecasting stock return volatility: The role of shrinkage approaches in a data‐rich environment," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 980-996, August.
    13. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    14. Christopher J. Neely & David E. Rapach & Jun Tu & Guofu Zhou, 2014. "Forecasting the Equity Risk Premium: The Role of Technical Indicators," Management Science, INFORMS, vol. 60(7), pages 1772-1791, July.
    15. Kim Christensen & Mathias Siggaard & Bezirgen Veliyev, 2023. "A Machine Learning Approach to Volatility Forecasting," Journal of Financial Econometrics, Oxford University Press, vol. 21(5), pages 1680-1727.
    16. Yin, Libo & Yang, Qingyuan, 2016. "Predicting the oil prices: Do technical indicators help?," Energy Economics, Elsevier, vol. 56(C), pages 338-350.
    17. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2024. "Machine-learning stock market volatility: Predictability, drivers, and economic value," International Review of Financial Analysis, Elsevier, vol. 94(C).
    18. Mittnik, Stefan & Robinzonov, Nikolay & Spindler, Martin, 2015. "Stock market volatility: Identifying major drivers and the nature of their impact," Journal of Banking & Finance, Elsevier, vol. 58(C), pages 1-14.
    19. DeMiguel, Victor & Gil-Bazo, Javier & Nogales, Francisco J. & Santos, André A.P., 2023. "Machine learning and fund characteristics help to select mutual funds with positive alpha," Journal of Financial Economics, Elsevier, vol. 150(3).
    20. Liu, Li & Pan, Zhiyuan, 2020. "Forecasting stock market volatility: The role of technical variables," Economic Modelling, Elsevier, vol. 84(C), pages 55-65.
    21. Nonejad, Nima, 2017. "Forecasting aggregate stock market volatility using financial and macroeconomic predictors: Which models forecast best, when and why?," Journal of Empirical Finance, Elsevier, vol. 42(C), pages 131-154.
    22. Patton, Andrew J., 2011. "Volatility forecast comparison using imperfect volatility proxies," Journal of Econometrics, Elsevier, vol. 160(1), pages 246-256, January.
    23. Ma, Feng & Liu, Jing & Wahab, M.I.M. & Zhang, Yaojie, 2018. "Forecasting the aggregate oil price volatility in a data-rich environment," Economic Modelling, Elsevier, vol. 72(C), pages 320-332.
    24. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    25. Li Liu & Feng Ma & Qing Zeng & Yaojie Zhang, 2020. "Forecasting the aggregate stock market volatility in a data-rich world," Applied Economics, Taylor & Francis Journals, vol. 52(32), pages 3448-3463, June.
    26. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    27. Zhang, Yaojie & Wei, Yu & Zhang, Yi & Jin, Daxiang, 2019. "Forecasting oil price volatility: Forecast combination versus shrinkage method," Energy Economics, Elsevier, vol. 80(C), pages 423-433.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2024. "Machine-learning stock market volatility: Predictability, drivers, and economic value," International Review of Financial Analysis, Elsevier, vol. 94(C).
    2. Zhang, Yaojie & Wahab, M.I.M. & Wang, Yudong, 2023. "Forecasting crude oil market volatility using variable selection and common factor," International Journal of Forecasting, Elsevier, vol. 39(1), pages 486-502.
    3. Gong, Xue & Ye, Xin & Zhang, Weiguo & Zhang, Yue, 2023. "Predicting energy futures high-frequency volatility using technical indicators: The role of interaction," Energy Economics, Elsevier, vol. 119(C).
    4. Wen, Danyan & He, Mengxi & Wang, Yudong & Zhang, Yaojie, 2024. "Forecasting crude oil market volatility: A comprehensive look at uncertainty variables," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1022-1041.
    5. Li, Xiaodan & Gong, Xue & Ge, Futing & Huang, Jingjing, 2024. "Forecasting stock volatility using pseudo-out-of-sample information," International Review of Economics & Finance, Elsevier, vol. 90(C), pages 123-135.
    6. He, Mengxi & Wang, Yudong & Zeng, Qing & Zhang, Yaojie, 2023. "Forecasting aggregate stock market volatility with industry volatilities: The role of spillover index," Research in International Business and Finance, Elsevier, vol. 65(C).
    7. Zhu, Haibin & Bai, Lu & He, Lidan & Liu, Zhi, 2023. "Forecasting realized volatility with machine learning: Panel data perspective," Journal of Empirical Finance, Elsevier, vol. 73(C), pages 251-271.
    8. He, Mengxi & Wen, Danyan & Xing, Lu & Zhang, Yaojie, 2024. "Industry volatility concentration and the predictability of aggregate stock market volatility," International Review of Economics & Finance, Elsevier, vol. 95(C).
    9. Niu, Zibo & Wang, Chenlu & Zhang, Hongwei, 2023. "Forecasting stock market volatility with various geopolitical risks categories: New evidence from machine learning models," International Review of Financial Analysis, Elsevier, vol. 89(C).
    10. Zhang, Hongwei & Wang, Wentao & Niu, Zibo, 2024. "Geopolitical risks and crude oil futures volatility: Evidence from machine learning," Resources Policy, Elsevier, vol. 98(C).
    11. Mengxi He & Xianfeng Hao & Yaojie Zhang & Fanyi Meng, 2021. "Forecasting stock return volatility using a robust regression model," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(8), pages 1463-1478, December.
    12. Lu, Xinjie & Ma, Feng & Xu, Jin & Zhang, Zehui, 2022. "Oil futures volatility predictability: New evidence based on machine learning models11All the authors contribute to the paper equally," International Review of Financial Analysis, Elsevier, vol. 83(C).
    13. Guo, Yangli & Ma, Feng & Li, Haibo & Lai, Xiaodong, 2022. "Oil price volatility predictability based on global economic conditions," International Review of Financial Analysis, Elsevier, vol. 82(C).
    14. Niu, Zibo & Demirer, Riza & Suleman, Muhammad Tahir & Zhang, Hongwei & Zhu, Xuehong, 2024. "Do industries predict stock market volatility? Evidence from machine learning models," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 90(C).
    15. Zhifeng Dai & Tingyu Li & Mi Yang, 2022. "Forecasting stock return volatility: The role of shrinkage approaches in a data‐rich environment," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 980-996, August.
    16. Zeng, Qing & Lu, Xinjie & Xu, Jin & Lin, Yu, 2024. "Macro-Driven Stock Market Volatility Prediction: Insights from a New Hybrid Machine Learning Approach," International Review of Financial Analysis, Elsevier, vol. 96(PB).
    17. Zhang, Hongwei & Zhao, Xinyi & Gao, Wang & Niu, Zibo, 2023. "The role of higher moments in predicting China's oil futures volatility: Evidence from machine learning models," Journal of Commodity Markets, Elsevier, vol. 32(C).
    18. Danyan Wen & Mengxi He & Yaojie Zhang & Yudong Wang, 2022. "Forecasting realized volatility of Chinese stock market: A simple but efficient truncated approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(2), pages 230-251, March.
    19. Guo, Yangli & He, Feng & Liang, Chao & Ma, Feng, 2022. "Oil price volatility predictability: New evidence from a scaled PCA approach," Energy Economics, Elsevier, vol. 105(C).
    20. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2023. "Gold risk premium estimation with machine learning methods," Journal of Commodity Markets, Elsevier, vol. 31(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:jforec:v:44:y:2025:i:4:p:1383-1402. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.