IDEAS home Printed from https://ideas.repec.org/a/eee/jrpoli/v81y2023ics0301420723000715.html
   My bibliography  Save this article

Crude oil price prediction using deep reinforcement learning

Author

Listed:
  • Liang, Xuedong
  • Luo, Peng
  • Li, Xiaoyan
  • Wang, Xia
  • Shu, Lingli

Abstract

Crude oil price forecasting has received considerable attention owing to its significance in the commodity market and non-linear complexity in forecasting tasks. This study aims to develop a novel deep reinforcement learning algorithm for multi-step ahead crude oil price forecasting in three major commodity exchanges. The proposed algorithm includes two main improvements: (a) A dynamic action exploration mechanism based on the stochastic processes conforming to commodity price fluctuations is designed for accuracy and generalization. (b) A dynamic update policy of network parameters based on approximate optimization theory is developed to improve the network's learning efficiency. The algorithm's effectiveness is experimentally verified and compared with five state-of-the-art algorithms. The main findings are as follows. (a) DRL's forecasting ability is developed in crude oil price forecasting, which may be extended to the forecasting of other natural resource prices. (b) The proposed algorithm can be applied to the data of the world's three major crude oil price benchmarks with considerable universality. (c) The accuracy of the proposed algorithm declines indistinctively with the expansion of the forecasting step; however, it reflects the actual price and fluctuation. These findings have implications in accelerating the global economic recovery and exploring AI in the energy market.

Suggested Citation

  • Liang, Xuedong & Luo, Peng & Li, Xiaoyan & Wang, Xia & Shu, Lingli, 2023. "Crude oil price prediction using deep reinforcement learning," Resources Policy, Elsevier, vol. 81(C).
  • Handle: RePEc:eee:jrpoli:v:81:y:2023:i:c:s0301420723000715
    DOI: 10.1016/j.resourpol.2023.103363
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0301420723000715
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.resourpol.2023.103363?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Marchese, Malvina & Kyriakou, Ioannis & Tamvakis, Michael & Di Iorio, Francesca, 2020. "Forecasting crude oil and refined products volatilities and correlations: New evidence from fractionally integrated multivariate GARCH models," Energy Economics, Elsevier, vol. 88(C).
    2. Wang, Jue & Zhou, Hao & Hong, Tao & Li, Xiang & Wang, Shouyang, 2020. "A multi-granularity heterogeneous combination approach to crude oil price forecasting," Energy Economics, Elsevier, vol. 91(C).
    3. Stavroula P. Fameliti & Vasiliki D. Skintzi, 2022. "Statistical and economic performance of combination methods for forecasting crude oil price volatility," Applied Economics, Taylor & Francis Journals, vol. 54(26), pages 3031-3054, June.
    4. Atukeren, Erdal & Çevik, Emrah İsmail & Korkmaz, Turhan, 2021. "Volatility spillovers between WTI and Brent spot crude oil prices: an analysis of granger causality in variance patterns over time," Research in International Business and Finance, Elsevier, vol. 56(C).
    5. Urolagin, Siddhaling & Sharma, Nikhil & Datta, Tapan Kumar, 2021. "A combined architecture of multivariate LSTM with Mahalanobis and Z-Score transformations for oil price forecasting," Energy, Elsevier, vol. 231(C).
    6. Jin, Xiaoye & Xiaowen Lin, Sharon & Tamvakis, Michael, 2012. "Volatility transmission and volatility impulse response functions in crude oil markets," Energy Economics, Elsevier, vol. 34(6), pages 2125-2134.
    7. Liu, Wenwen & Chen, Xue, 2022. "Natural resources commodity prices volatility and economic uncertainty: Evaluating the role of oil and gas rents in COVID-19," Resources Policy, Elsevier, vol. 76(C).
    8. Gkillas, Konstantinos & Gupta, Rangan & Pierdzioch, Christian, 2020. "Forecasting realized oil-price volatility: The role of financial stress and asymmetric loss," Journal of International Money and Finance, Elsevier, vol. 104(C).
    9. Michael L. Littman, 2015. "Reinforcement learning improves behaviour from evaluative feedback," Nature, Nature, vol. 521(7553), pages 445-451, May.
    10. Sun, Chuanwang & Zhan, Yanhong & Peng, Yiqi & Cai, Weiyi, 2022. "Crude oil price and exchange rate: Evidence from the period before and after the launch of China's crude oil futures," Energy Economics, Elsevier, vol. 105(C).
    11. Liu, Siyao & Fang, Wei & Gao, Xiangyun & An, Feng & Jiang, Meihui & Li, Yang, 2019. "Long-term memory dynamics of crude oil price spread in non-dollar countries under the influence of exchange rates," Energy, Elsevier, vol. 182(C), pages 753-764.
    12. Kisswani, Khalid M. & Nusair, Salah A., 2013. "Non-linearities in the dynamics of oil prices," Energy Economics, Elsevier, vol. 36(C), pages 341-353.
    13. Lv, Wendai & Wu, Qian, 2022. "Global economic conditions index and oil price predictability," Finance Research Letters, Elsevier, vol. 48(C).
    14. Zhang, Tingting & Tang, Zhenpeng & Wu, Junchuan & Du, Xiaoxu & Chen, Kaijie, 2021. "Multi-step-ahead crude oil price forecasting based on two-layer decomposition technique and extreme learning machine optimized by the particle swarm optimization algorithm," Energy, Elsevier, vol. 229(C).
    15. Boz, Emine, 2011. "Sovereign default, private sector creditors, and the IFIs," Journal of International Economics, Elsevier, vol. 83(1), pages 70-82, January.
    16. Omar, Ayman M.A. & Wisniewski, Tomasz Piotr & Nolte, Sandra, 2017. "Diversifying away the risk of war and cross-border political crisis," Energy Economics, Elsevier, vol. 64(C), pages 494-510.
    17. Zhang, Hong & Nguyen, Hoang & Bui, Xuan-Nam & Pradhan, Biswajeet & Mai, Ngoc-Luan & Vu, Diep-Anh, 2021. "Proposing two novel hybrid intelligence models for forecasting copper price based on extreme learning machine and meta-heuristic algorithms," Resources Policy, Elsevier, vol. 73(C).
    18. Lizardo, Radhamés A. & Mollick, André V., 2010. "Oil price fluctuations and U.S. dollar exchange rates," Energy Economics, Elsevier, vol. 32(2), pages 399-408, March.
    19. Shehabi, Manal, 2022. "Modeling long-term impacts of the COVID-19 pandemic and oil price declines on Gulf oil economies," Economic Modelling, Elsevier, vol. 112(C).
    20. Rabeh Khalfaoui & Sakiru Adebola Solarin & Adel Al-Qadasi & Sami Ben Jabeur, 2022. "Dynamic causality interplay from COVID-19 pandemic to oil price, stock market, and economic policy uncertainty: evidence from oil-importing and oil-exporting countries," Annals of Operations Research, Springer, vol. 313(1), pages 105-143, June.
    21. Manickavasagam, Jeevananthan & Visalakshmi, S. & Apergis, Nicholas, 2020. "A novel hybrid approach to forecast crude oil futures using intraday data," Technological Forecasting and Social Change, Elsevier, vol. 158(C).
    22. Tiwari, Aviral Kumar & Kumar, Satish & Pathak, Rajesh & Roubaud, David, 2019. "Testing the oil price efficiency using various measures of long-range dependence," Energy Economics, Elsevier, vol. 84(C).
    23. Tang, Ling & Wu, Yao & Yu, Lean, 2018. "A randomized-algorithm-based decomposition-ensemble learning methodology for energy price forecasting," Energy, Elsevier, vol. 157(C), pages 526-538.
    24. Demirer, Riza & Gupta, Rangan & Pierdzioch, Christian & Shahzad, Syed Jawad Hussain, 2020. "The predictive power of oil price shocks on realized volatility of oil: A note," Resources Policy, Elsevier, vol. 69(C).
    25. Kayalar, Derya Ezgi & Küçüközmen, C. Coşkun & Selcuk-Kestel, A. Sevtap, 2017. "The impact of crude oil prices on financial market indicators: copula approach," Energy Economics, Elsevier, vol. 61(C), pages 162-173.
    26. Scheitrum, Daniel P. & Carter, Colin A. & Revoredo-Giha, Cesar, 2018. "WTI and Brent futures pricing structure," Energy Economics, Elsevier, vol. 72(C), pages 462-469.
    27. Chen, Wang & Ma, Feng & Wei, Yu & Liu, Jing, 2020. "Forecasting oil price volatility using high-frequency data: New evidence," International Review of Economics & Finance, Elsevier, vol. 66(C), pages 1-12.
    28. Huang, Xiaohong & Huang, Shupei, 2020. "Identifying the comovement of price between China's and international crude oil futures: A time-frequency perspective," International Review of Financial Analysis, Elsevier, vol. 72(C).
    29. Alqahtani, Abdullah & Klein, Tony & Khalid, Ali, 2019. "The impact of oil price uncertainty on GCC stock markets," Resources Policy, Elsevier, vol. 64(C).
    30. Li, Ranran & Hu, Yucai & Heng, Jiani & Chen, Xueli, 2021. "A novel multiscale forecasting model for crude oil price time series," Technological Forecasting and Social Change, Elsevier, vol. 173(C).
    31. Vacca, Gianmarco & Zoia, Maria Grazia & Bagnato, Luca, 2022. "Forecasting in GARCH models with polynomially modified innovations," International Journal of Forecasting, Elsevier, vol. 38(1), pages 117-141.
    32. Noguera-Santaella, José, 2016. "Geopolitics and the oil price," Economic Modelling, Elsevier, vol. 52(PB), pages 301-309.
    33. David Bourghelle & Fredj Jawadi & Philippe Rozin, 2021. "Oil price volatility in the context of Covid-19," International Economics, CEPII research center, issue 167, pages 39-49.
    34. Wang, Lu & Ma, Feng & Hao, Jianyang & Gao, Xinxin, 2021. "Forecasting crude oil volatility with geopolitical risk: Do time-varying switching probabilities play a role?," International Review of Financial Analysis, Elsevier, vol. 76(C).
    35. Rehrl, Tobias & Friedrich, Rainer, 2006. "Modelling long-term oil price and extraction with a Hubbert approach: The LOPEX model," Energy Policy, Elsevier, vol. 34(15), pages 2413-2428, October.
    36. Volodymyr Mnih & Koray Kavukcuoglu & David Silver & Andrei A. Rusu & Joel Veness & Marc G. Bellemare & Alex Graves & Martin Riedmiller & Andreas K. Fidjeland & Georg Ostrovski & Stig Petersen & Charle, 2015. "Human-level control through deep reinforcement learning," Nature, Nature, vol. 518(7540), pages 529-533, February.
    37. Rabeh Khalfaoui & Sakiru Adebola Solarin & Adel Al-Qadasi & Sami Ben Jabeur, 2022. "Dynamic causality interplay from COVID-19 pandemic to oil price, stock market, and economic policy uncertainty: evidence from oil-importing and oil-exporting countries," Annals of Operations Research, Springer, vol. 313(1), pages 105-143, June.
    38. Mastroeni, Loretta & Mazzoccoli, Alessandro & Quaresima, Greta & Vellucci, Pierluigi, 2021. "Decoupling and recoupling in the crude oil price benchmarks: An investigation of similarity patterns," Energy Economics, Elsevier, vol. 94(C).
    39. Le, Thai-Ha & Le, Anh Tu & Le, Ha-Chi, 2021. "The historic oil price fluctuation during the Covid-19 pandemic: What are the causes?," Research in International Business and Finance, Elsevier, vol. 58(C).
    40. Bourghelle, David & Jawadi, Fredj & Rozin, Philippe, 2021. "Oil price volatility in the context of Covid-19," International Economics, Elsevier, vol. 167(C), pages 39-49.
    41. Lin, Yu & Xiao, Yang & Li, Fuxing, 2020. "Forecasting crude oil price volatility via a HM-EGARCH model," Energy Economics, Elsevier, vol. 87(C).
    42. Bouoiyour, Jamal & Selmi, Refk & Hammoudeh, Shawkat & Wohar, Mark E., 2019. "What are the categories of geopolitical risks that could drive oil prices higher? Acts or threats?," Energy Economics, Elsevier, vol. 84(C).
    43. Joo, Young C. & Park, Sung Y., 2017. "Oil prices and stock markets: Does the effect of uncertainty change over time?," Energy Economics, Elsevier, vol. 61(C), pages 42-51.
    44. Zulfigarov, Farid & Neuenkirch, Matthias, 2020. "The impact of oil price changes on selected macroeconomic indicators in Azerbaijan," Economic Systems, Elsevier, vol. 44(4).
    45. Bai, Yun & Li, Xixi & Yu, Hao & Jia, Suling, 2022. "Crude oil price forecasting incorporating news text," International Journal of Forecasting, Elsevier, vol. 38(1), pages 367-383.
    46. Ignatieva, Katja & Wong, Patrick, 2022. "Modelling high frequency crude oil dynamics using affine and non-affine jump–diffusion models," Energy Economics, Elsevier, vol. 108(C).
    47. Zhou, Xinlei & Lin, Wenye & Kumar, Ritunesh & Cui, Ping & Ma, Zhenjun, 2022. "A data-driven strategy using long short term memory models and reinforcement learning to predict building electricity consumption," Applied Energy, Elsevier, vol. 306(PB).
    48. Mann, Janelle & Sephton, Peter, 2016. "Global relationships across crude oil benchmarks," Journal of Commodity Markets, Elsevier, vol. 2(1), pages 1-5.
    49. Julien Chevallier & Bangzhu Zhu & Lyuyuan Zhang, 2021. "Forecasting Inflection Points: Hybrid Methods with Multiscale Machine Learning Algorithms," Computational Economics, Springer;Society for Computational Economics, vol. 57(2), pages 537-575, February.
    50. Guliyev, Hasraddin & Mustafayev, Eldayag, 2022. "Predicting the changes in the WTI crude oil price dynamics using machine learning models," Resources Policy, Elsevier, vol. 77(C).
    51. Khan, Khalid & Su, Chi-Wei & Umar, Muhammad & Yue, Xiao-Guang, 2021. "Do crude oil price bubbles occur?," Resources Policy, Elsevier, vol. 71(C).
    52. Movagharnejad, Kamyar & Mehdizadeh, Bahman & Banihashemi, Morteza & Kordkheili, Masoud Sheikhi, 2011. "Forecasting the differences between various commercial oil prices in the Persian Gulf region by neural network," Energy, Elsevier, vol. 36(7), pages 3979-3984.
    53. Zhao, Wan-Li & Fan, Ying & Ji, Qiang, 2022. "Extreme risk spillover between crude oil price and financial factors," Finance Research Letters, Elsevier, vol. 46(PA).
    54. Ma, Richie Ruchuan & Xiong, Tao & Bao, Yukun, 2021. "The Russia-Saudi Arabia oil price war during the COVID-19 pandemic," Energy Economics, Elsevier, vol. 102(C).
    55. Wang, Jue & Athanasopoulos, George & Hyndman, Rob J. & Wang, Shouyang, 2018. "Crude oil price forecasting based on internet concern using an extreme learning machine," International Journal of Forecasting, Elsevier, vol. 34(4), pages 665-677.
    56. Chai, Jian & Guo, Ju-E. & Meng, Lei & Wang, Shou-Yang, 2011. "Exploring the core factors and its dynamic effects on oil price: An application on path analysis and BVAR-TVP model," Energy Policy, Elsevier, vol. 39(12), pages 8022-8036.
    57. Duan, Huiming & Liu, Yunmei & Wang, Guan, 2022. "A novel dynamic time-delay grey model of energy prices and its application in crude oil price forecasting," Energy, Elsevier, vol. 251(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xu, Kunliang & Wang, Weiqing, 2023. "Limited information limits accuracy: Whether ensemble empirical mode decomposition improves crude oil spot price prediction?," International Review of Financial Analysis, Elsevier, vol. 87(C).
    2. Fiszeder, Piotr & Fałdziński, Marcin & Molnár, Peter, 2023. "Attention to oil prices and its impact on the oil, gold and stock markets and their covariance," Energy Economics, Elsevier, vol. 120(C).
    3. Song, Yixuan & He, Mengxi & Wang, Yudong & Zhang, Yaojie, 2022. "Forecasting crude oil market volatility: A newspaper-based predictor regarding petroleum market volatility," Resources Policy, Elsevier, vol. 79(C).
    4. Xiao, Jihong & Wen, Fenghua & He, Zhifang, 2023. "Impact of geopolitical risks on investor attention and speculation in the oil market: Evidence from nonlinear and time-varying analysis," Energy, Elsevier, vol. 267(C).
    5. Hao, Jun & Feng, Qianqian & Yuan, Jiaxin & Sun, Xiaolei & Li, Jianping, 2022. "A dynamic ensemble learning with multi-objective optimization for oil prices prediction," Resources Policy, Elsevier, vol. 79(C).
    6. Jiang, He & Hu, Weiqiang & Xiao, Ling & Dong, Yao, 2022. "A decomposition ensemble based deep learning approach for crude oil price forecasting," Resources Policy, Elsevier, vol. 78(C).
    7. Chatziantoniou, Ioannis & Elsayed, Ahmed H. & Gabauer, David & Gozgor, Giray, 2023. "Oil price shocks and exchange rate dynamics: Evidence from decomposed and partial connectedness measures for oil importing and exporting economies," Energy Economics, Elsevier, vol. 120(C).
    8. Wu, Chunying & Wang, Jianzhou & Hao, Yan, 2022. "Deterministic and uncertainty crude oil price forecasting based on outlier detection and modified multi-objective optimization algorithm," Resources Policy, Elsevier, vol. 77(C).
    9. Wang, Xuerui & Li, Xiangyu & Li, Shaoting, 2022. "Point and interval forecasting system for crude oil price based on complete ensemble extreme-point symmetric mode decomposition with adaptive noise and intelligent optimization algorithm," Applied Energy, Elsevier, vol. 328(C).
    10. Chatziantoniou, Ioannis & Gabauer, David & Perez de Gracia, Fernando, 2022. "Tail risk connectedness in the refined petroleum market: A first look at the impact of the COVID-19 pandemic," Energy Economics, Elsevier, vol. 111(C).
    11. Liu, Yang & Han, Liyan & Xu, Yang, 2021. "The impact of geopolitical uncertainty on energy volatility," International Review of Financial Analysis, Elsevier, vol. 75(C).
    12. Naif Alsagr & Stefan F. Van Hemmen Almazor, 2020. "Oil Rent, Geopolitical Risk and Banking Sector Performance," International Journal of Energy Economics and Policy, Econjournals, vol. 10(5), pages 305-314.
    13. Daniel Stefan Armeanu & Stefan Cristian Gherghina & Jean Vasile Andrei & Camelia Catalina Joldes, 2023. "Evidence from the nonlinear autoregressive distributed lag model on the asymmetric influence of the first wave of the COVID-19 pandemic on energy markets," Energy & Environment, , vol. 34(5), pages 1433-1470, August.
    14. Huang, Jionghao & Li, Ziruo & Xia, Xiaohua, 2021. "Network diffusion of international oil volatility risk in China's stock market: Quantile interconnectedness modelling and shock decomposition analysis," International Review of Economics & Finance, Elsevier, vol. 76(C), pages 1-39.
    15. Gao, Xin & Li, Bingxin & Liu, Rui, 2023. "The relative pricing of WTI and Brent crude oil futures: Expectations or risk premia?," Journal of Commodity Markets, Elsevier, vol. 30(C).
    16. Taicir Mezghani & Mouna Boujelbène Abbes, 2023. "Forecast the Role of GCC Financial Stress on Oil Market and GCC Financial Markets Using Convolutional Neural Networks," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 30(3), pages 505-530, September.
    17. Tunc, Ahmet & Kocoglu, Mustafa & Aslan, Alper, 2022. "Time-varying characteristics of the simultaneous interactions between economic uncertainty, international oil prices and GDP: A novel approach for Germany," Resources Policy, Elsevier, vol. 77(C).
    18. Li, Mingchen & Cheng, Zishu & Lin, Wencan & Wei, Yunjie & Wang, Shouyang, 2023. "What can be learned from the historical trend of crude oil prices? An ensemble approach for crude oil price forecasting," Energy Economics, Elsevier, vol. 123(C).
    19. Apostolakis, George N. & Floros, Christos & Gkillas, Konstantinos & Wohar, Mark, 2021. "Financial stress, economic policy uncertainty, and oil price uncertainty," Energy Economics, Elsevier, vol. 104(C).
    20. Shi, Tao & Li, Chongyang & Zhang, Wei & Zhang, Yi, 2023. "Forecasting on metal resource spot settlement price: New evidence from the machine learning model," Resources Policy, Elsevier, vol. 81(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jrpoli:v:81:y:2023:i:c:s0301420723000715. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/30467 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.