IDEAS home Printed from https://ideas.repec.org/a/ris/apltrx/0457.html
   My bibliography  Save this article

Forecasting oil prices with penalized regressions, variance risk premia and Google data

Author

Listed:
  • Lycheva, Maria

    (ICEF, HSE university, Moscow;)

  • Mironenkov, Alexey

    (Moscow School of Economics, Moscow State University)

  • Kurbatskii, Alexey

    (Moscow School of Economics, Moscow State University)

  • Fantazzini, Dean

    (Moscow School of Economics, Moscow State University)

Abstract

This paper investigates whether augmenting models with the variance risk premium (VRP) and Google search data improves the quality of the forecasts for real oil prices. We considered a time sample of monthly data from 2007 to 2019 that includes several episodes of high volatility in the oil market. Our evidence shows that penalized regressions provided the best forecasting performances across most of the forecasting horizons. Moreover, we found that models using the VRP as an additional predictor performed best for forecasts up to 6–12 months ahead forecasts, while models using Google data as an additional predictor performed better for longer-term forecasts up to 12–24 months ahead. However, we found that the differences in forecasting performances were not statistically different for most models, and only the Principal Component Regression (PCR) and the Partial least squares (PLS) regression were consistently excluded from the set of best forecasting models. These results also held after a set of robustness checks that considered model specifications using a wider set of influential variables, a Hierarchical Vector Auto-Regression model estimated with the LASSO, and a set of forecasting models using a simplified specification for Google Trends data

Suggested Citation

  • Lycheva, Maria & Mironenkov, Alexey & Kurbatskii, Alexey & Fantazzini, Dean, 2022. "Forecasting oil prices with penalized regressions, variance risk premia and Google data," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 68, pages 28-49.
  • Handle: RePEc:ris:apltrx:0457
    as

    Download full text from publisher

    File URL: http://pe.cemi.rssi.ru/pe_2022_68_028-049.pdf
    File Function: Full text
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lutz Kilian, 2016. "The Impact of the Shale Oil Revolution on U.S. Oil and Gasoline Prices," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 10(2), pages 185-205.
    2. Alquist, Ron & Kilian, Lutz & Vigfusson, Robert J., 2013. "Forecasting the Price of Oil," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 427-507, Elsevier.
    3. repec:dau:papers:123456789/11714 is not listed on IDEAS
    4. Tim Bollerslev & George Tauchen & Hao Zhou, 2009. "Expected Stock Returns and Variance Risk Premia," Review of Financial Studies, Society for Financial Studies, vol. 22(11), pages 4463-4492, November.
    5. Renée Fry & Adrian Pagan, 2011. "Sign Restrictions in Structural Vector Autoregressions: A Critical Review," Journal of Economic Literature, American Economic Association, vol. 49(4), pages 938-960, December.
    6. Julien, Chevallier & Sévi, Benoît, 2013. "A Fear Index to Predict Oil Futures Returns," Energy: Resources and Markets 156489, Fondazione Eni Enrico Mattei (FEEM).
    7. Zhang, Yaojie & Ma, Feng & Wang, Yudong, 2019. "Forecasting crude oil prices with a large set of predictors: Can LASSO select powerful predictors?," Journal of Empirical Finance, Elsevier, vol. 54(C), pages 97-117.
    8. Christiane Baumeister & Lutz Kilian, 2015. "Forecasting the Real Price of Oil in a Changing World: A Forecast Combination Approach," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(3), pages 338-351, July.
    9. Dean Fantazzini & Nikita Fomichev, 2014. "Forecasting the real price of oil using online search data," International Journal of Computational Economics and Econometrics, Inderscience Enterprises Ltd, vol. 4(1/2), pages 4-31.
    10. Bollerslev, Tim & Marrone, James & Xu, Lai & Zhou, Hao, 2014. "Stock Return Predictability and Variance Risk Premia: Statistical Inference and International Evidence," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 49(3), pages 633-661, June.
    11. Qadan, Mahmoud & Nama, Hazar, 2018. "Investor sentiment and the price of oil," Energy Economics, Elsevier, vol. 69(C), pages 42-58.
    12. Hsiao, Cheng & Wan, Shui Ki, 2014. "Is there an optimal forecast combination?," Journal of Econometrics, Elsevier, vol. 178(P2), pages 294-309.
    13. T. Bazhenov & D. Fantazzini, 2019. "Forecasting Realized Volatility of Russian stocks using Google Trends and Implied Volatility," Russian Journal of Industrial Economics, MISIS, vol. 12(1).
    14. Fantazzini, Dean & Toktamysova, Zhamal, 2015. "Forecasting German car sales using Google data and multivariate models," International Journal of Production Economics, Elsevier, vol. 170(PA), pages 97-135.
    15. Haas Ornelas, José Renato, 2019. "Expected currency returns and volatility risk premia," The North American Journal of Economics and Finance, Elsevier, vol. 49(C), pages 206-234.
    16. Fantazzini, Dean, 2016. "The oil price crash in 2014/15: Was there a (negative) financial bubble?," Energy Policy, Elsevier, vol. 96(C), pages 383-396.
    17. Lutz Kilian, 2008. "The Economic Effects of Energy Price Shocks," Journal of Economic Literature, American Economic Association, vol. 46(4), pages 871-909, December.
    18. Hamilton, James D., 2011. "Nonlinearities And The Macroeconomic Effects Of Oil Prices," Macroeconomic Dynamics, Cambridge University Press, vol. 15(S3), pages 364-378, November.
    19. Adem Atmaz, 2022. "Stock Return Extrapolation, Option Prices, and Variance Risk Premium," Review of Financial Studies, Society for Financial Studies, vol. 35(3), pages 1348-1393.
    20. Timmermann, Allan, 2006. "Forecast Combinations," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 4, pages 135-196, Elsevier.
    21. Afkhami, Mohamad & Cormack, Lindsey & Ghoddusi, Hamed, 2017. "Google search keywords that best predict energy price volatility," Energy Economics, Elsevier, vol. 67(C), pages 17-27.
    22. James D. Hamilton, 2009. "Understanding Crude Oil Prices," The Energy Journal, International Association for Energy Economics, vol. 0(Number 2), pages 179-206.
    23. Ornelas, José Renato Haas & Mauad, Roberto Baltieri, 2019. "Volatility risk premia and future commodity returns," Journal of International Money and Finance, Elsevier, vol. 96(C), pages 341-360.
    24. Capistrán, Carlos & Timmermann, Allan, 2009. "Forecast Combination With Entry and Exit of Experts," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 428-440.
    25. Lutz Kilian, 2009. "Not All Oil Price Shocks Are Alike: Disentangling Demand and Supply Shocks in the Crude Oil Market," American Economic Review, American Economic Association, vol. 99(3), pages 1053-1069, June.
    26. Christopher J. Neely & David E. Rapach & Jun Tu & Guofu Zhou, 2014. "Forecasting the Equity Risk Premium: The Role of Technical Indicators," Management Science, INFORMS, vol. 60(7), pages 1772-1791, July.
    27. Londono, Juan M. & Zhou, Hao, 2017. "Variance risk premiums and the forward premium puzzle," Journal of Financial Economics, Elsevier, vol. 124(2), pages 415-440.
    28. Fantazzini, Dean & Höök, Mikael & Angelantoni, André, 2011. "Global oil risks in the early 21st century," Energy Policy, Elsevier, vol. 39(12), pages 7865-7873.
    29. Miao, Hong & Ramchander, Sanjay & Wang, Tianyang & Yang, Dongxiao, 2017. "Influential factors in crude oil price forecasting," Energy Economics, Elsevier, vol. 68(C), pages 77-88.
    30. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    31. Kaufmann, Robert K. & Dees, Stephane & Gasteuil, Audrey & Mann, Michael, 2008. "Oil prices: The role of refinery utilization, futures markets and non-linearities," Energy Economics, Elsevier, vol. 30(5), pages 2609-2622, September.
    32. Christiane Baumeister & James D. Hamilton, 2019. "Structural Interpretation of Vector Autoregressions with Incomplete Identification: Revisiting the Role of Oil Supply and Demand Shocks," American Economic Review, American Economic Association, vol. 109(5), pages 1873-1910, May.
    33. Fantazzini, Dean & Shangina, Tamara, 2019. "The importance of being informed: forecasting market risk measures for the Russian RTS index future using online data and implied volatility over two decades," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 55, pages 5-31.
    34. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    35. Lutz Kilian & Daniel P. Murphy, 2014. "The Role Of Inventories And Speculative Trading In The Global Market For Crude Oil," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 29(3), pages 454-478, April.
    36. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1.
    37. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xing, Li-Min & Zhang, Yue-Jun, 2022. "Forecasting crude oil prices with shrinkage methods: Can nonconvex penalty and Huber loss help?," Energy Economics, Elsevier, vol. 110(C).
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Wang, Yudong & Hao, Xianfeng, 2023. "Forecasting the real prices of crude oil: What is the role of parameter instability?," Energy Economics, Elsevier, vol. 117(C).
    4. Dominik Wolff & Ulrich Neugebauer, 2019. "Tree-based machine learning approaches for equity market predictions," Journal of Asset Management, Palgrave Macmillan, vol. 20(4), pages 273-288, July.
    5. Costa, Alexandre Bonnet R. & Ferreira, Pedro Cavalcanti G. & Gaglianone, Wagner P. & Guillén, Osmani Teixeira C. & Issler, João Victor & Lin, Yihao, 2021. "Machine learning and oil price point and density forecasting," Energy Economics, Elsevier, vol. 102(C).
    6. Lu, Xinjie & Ma, Feng & Xu, Jin & Zhang, Zehui, 2022. "Oil futures volatility predictability: New evidence based on machine learning models11All the authors contribute to the paper equally," International Review of Financial Analysis, Elsevier, vol. 83(C).
    7. Cheng, Xian & Wu, Peng & Liao, Stephen Shaoyi & Wang, Xuelian, 2023. "An integrated model for crude oil forecasting: Causality assessment and technical efficiency," Energy Economics, Elsevier, vol. 117(C).
    8. Dbouk, Wassim & Jamali, Ibrahim, 2018. "Predicting daily oil prices: Linear and non-linear models," Research in International Business and Finance, Elsevier, vol. 46(C), pages 149-165.
    9. Baumeister, Christiane & Kilian, Lutz & Lee, Thomas K., 2014. "Are there gains from pooling real-time oil price forecasts?," Energy Economics, Elsevier, vol. 46(S1), pages 33-43.
    10. Liu, Li & Ma, Feng & Wang, Yudong, 2015. "Forecasting excess stock returns with crude oil market data," Energy Economics, Elsevier, vol. 48(C), pages 316-324.
    11. Francesco Ravazzolo & Joaquin L. Vespignani, 2015. "A new monthly indicator of global real economic activity," Globalization Institute Working Papers 244, Federal Reserve Bank of Dallas.
    12. Kilian, Lutz & Zhou, Xiaoqing, 2018. "Structural Interpretation of Vector Autoregressions with Incomplete Information: Revisiting the Role of Oil Supply and Demand S," CEPR Discussion Papers 13068, C.E.P.R. Discussion Papers.
    13. He, Mengxi & Zhang, Yaojie & Wen, Danyan & Wang, Yudong, 2021. "Forecasting crude oil prices: A scaled PCA approach," Energy Economics, Elsevier, vol. 97(C).
    14. Yin, Libo & Yang, Qingyuan, 2016. "Predicting the oil prices: Do technical indicators help?," Energy Economics, Elsevier, vol. 56(C), pages 338-350.
    15. Iason Kynigakis & Ekaterini Panopoulou, 2022. "Does model complexity add value to asset allocation? Evidence from machine learning forecasting models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(3), pages 603-639, April.
    16. Tian, Guangning & Peng, Yuchao & Meng, Yuhao, 2023. "Forecasting crude oil prices in the COVID-19 era: Can machine learn better?," Energy Economics, Elsevier, vol. 125(C).
    17. Lutz Kilian & Xiaoqing Zhou, 2018. "Structural Interpretation of Vector Autoregressions with Incomplete Identification: Revisiting the Role of Oil Supply and Demand Shocks: Comment," CESifo Working Paper Series 7166, CESifo.
    18. Knut Are Aastveit & Jamie L. Cross & Herman K. van Dijk, 2023. "Quantifying Time-Varying Forecast Uncertainty and Risk for the Real Price of Oil," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 41(2), pages 523-537, April.
    19. Wang, Yudong & Liu, Li & Wu, Chongfeng, 2017. "Forecasting the real prices of crude oil using forecast combinations over time-varying parameter models," Energy Economics, Elsevier, vol. 66(C), pages 337-348.
    20. Hao, Xianfeng & Zhao, Yuyang & Wang, Yudong, 2020. "Forecasting the real prices of crude oil using robust regression models with regularization constraints," Energy Economics, Elsevier, vol. 86(C).

    More about this item

    Keywords

    oil price; variance risk premium; Google Trends; VAR; LASSO; Ridge; Elastic Net; principal components; partial least squares;
    All these keywords.

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • C58 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Financial Econometrics
    • G17 - Financial Economics - - General Financial Markets - - - Financial Forecasting and Simulation
    • O13 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Agriculture; Natural Resources; Environment; Other Primary Products
    • Q47 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Energy - - - Energy Forecasting

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ris:apltrx:0457. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Anatoly Peresetsky (email available below). General contact details of provider: http://appliedeconometrics.cemi.rssi.ru/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.