IDEAS home Printed from https://ideas.repec.org/a/eee/empfin/v54y2019icp97-117.html
   My bibliography  Save this article

Forecasting crude oil prices with a large set of predictors: Can LASSO select powerful predictors?

Author

Listed:
  • Zhang, Yaojie
  • Ma, Feng
  • Wang, Yudong

Abstract

In this paper, we use two prevailing shrinkage methods, the lasso and elastic net, to predict oil price returns with a large set of predictors. The out-of-sample results indicate that the lasso and elastic net models outperform a host of widely used competing models in terms of out-of-sample R-square and success ratio. In an asset allocation exercise, a mean–variance investor obtains positive and sizeable economic gains based on the return forecasts of the lasso and elastic net methods relative to both the benchmark forecasts and competing forecasts. We further investigate the source of predictability from a variable selection perspective. The lasso and elastic net methods are found to select powerful predictors and the ones that can provide complementary information. The OLS regression models based on the selected predictors also exhibit better out-of-sample performances than the competing models. In addition, our results are robust to various settings.

Suggested Citation

  • Zhang, Yaojie & Ma, Feng & Wang, Yudong, 2019. "Forecasting crude oil prices with a large set of predictors: Can LASSO select powerful predictors?," Journal of Empirical Finance, Elsevier, vol. 54(C), pages 97-117.
  • Handle: RePEc:eee:empfin:v:54:y:2019:i:c:p:97-117
    DOI: 10.1016/j.jempfin.2019.08.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0927539819300726
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jempfin.2019.08.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Faria, Gonçalo & Verona, Fabio, 2018. "Forecasting stock market returns by summing the frequency-decomposed parts," Journal of Empirical Finance, Elsevier, vol. 45(C), pages 228-242.
    2. Ivo Welch & Amit Goyal, 2008. "A Comprehensive Look at The Empirical Performance of Equity Premium Prediction," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1455-1508, July.
    3. Baumeister, Christiane & Guérin, Pierre & Kilian, Lutz, 2015. "Do high-frequency financial data help forecast oil prices? The MIDAS touch at work," International Journal of Forecasting, Elsevier, vol. 31(2), pages 238-252.
    4. Yulei Luo & Eric R. Young, 2016. "Long‐Run Consumption Risk and Asset Allocation under Recursive Utility and Rational Inattention," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 48(2-3), pages 325-362, March.
    5. Zou, Hui, 2006. "The Adaptive Lasso and Its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1418-1429, December.
    6. Stivers, Adam, 2018. "Equity premium predictions with many predictors: A risk-based explanation of the size and value factors," Journal of Empirical Finance, Elsevier, vol. 45(C), pages 126-140.
    7. Jiahan Li & Ilias Tsiakas & Wei Wang, 2015. "Predicting Exchange Rates Out of Sample: Can Economic Fundamentals Beat the Random Walk?," The Journal of Financial Econometrics, Society for Financial Econometrics, vol. 13(2), pages 293-341.
    8. Gary Koop & Dimitris Korobilis, 2012. "Forecasting Inflation Using Dynamic Model Averaging," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 53(3), pages 867-886, August.
    9. Alquist, Ron & Kilian, Lutz & Vigfusson, Robert J., 2013. "Forecasting the Price of Oil," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 427-507, Elsevier.
    10. Xavier Gabaix, 2014. "A Sparsity-Based Model of Bounded Rationality," The Quarterly Journal of Economics, Oxford University Press, vol. 129(4), pages 1661-1710.
    11. Wang, Yudong & Liu, Li & Wu, Chongfeng, 2017. "Forecasting the real prices of crude oil using forecast combinations over time-varying parameter models," Energy Economics, Elsevier, vol. 66(C), pages 337-348.
    12. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    13. Pettenuzzo, Davide & Timmermann, Allan & Valkanov, Rossen, 2014. "Forecasting stock returns under economic constraints," Journal of Financial Economics, Elsevier, vol. 114(3), pages 517-553.
    14. Wang, Yudong & Liu, Li & Diao, Xundi & Wu, Chongfeng, 2015. "Forecasting the real prices of crude oil under economic and statistical constraints," Energy Economics, Elsevier, vol. 51(C), pages 599-608.
    15. Christiane Baumeister & Lutz Kilian, 2015. "Forecasting the Real Price of Oil in a Changing World: A Forecast Combination Approach," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(3), pages 338-351, July.
    16. Zhu, Xiaoneng & Zhu, Jie, 2013. "Predicting stock returns: A regime-switching combination approach and economic links," Journal of Banking & Finance, Elsevier, vol. 37(11), pages 4120-4133.
    17. Christiane Baumeister & Lutz Kilian, 2014. "What Central Bankers Need To Know About Forecasting Oil Prices," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 55, pages 869-889, August.
    18. Dashan Huang & Fuwei Jiang & Jun Tu & Guofu Zhou, 2015. "Investor Sentiment Aligned: A Powerful Predictor of Stock Returns," Review of Financial Studies, Society for Financial Studies, vol. 28(3), pages 791-837.
    19. Pesaran, M Hashem & Timmermann, Allan, 1992. "A Simple Nonparametric Test of Predictive Performance," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(4), pages 561-565, October.
    20. Lixin Huang & Hong Liu, 2007. "Rational Inattention and Portfolio Selection," Journal of Finance, American Finance Association, vol. 62(4), pages 1999-2040, August.
    21. Malcolm Baker & Jeffrey Wurgler, 2006. "Investor Sentiment and the Cross‐Section of Stock Returns," Journal of Finance, American Finance Association, vol. 61(4), pages 1645-1680, August.
    22. Baetje, Fabian & Menkhoff, Lukas, 2016. "Equity premium prediction: Are economic and technical indicators unstable?," International Journal of Forecasting, Elsevier, vol. 32(4), pages 1193-1207.
    23. Xavier Gabaix & David Laibson & Guillermo Moloche & Stephen Weinberg, 2006. "Costly Information Acquisition: Experimental Analysis of a Boundedly Rational Model," American Economic Review, American Economic Association, vol. 96(4), pages 1043-1068, September.
    24. Drachal, Krzysztof, 2016. "Forecasting spot oil price in a dynamic model averaging framework — Have the determinants changed over time?," Energy Economics, Elsevier, vol. 60(C), pages 35-46.
    25. Lutz Kilian & Cheolbeom Park, 2009. "The Impact Of Oil Price Shocks On The U.S. Stock Market," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 50(4), pages 1267-1287, November.
    26. Li, Jiahan & Tsiakas, Ilias, 2017. "Equity premium prediction: The role of economic and statistical constraints," Journal of Financial Markets, Elsevier, vol. 36(C), pages 56-75.
    27. Stambaugh, Robert F. & Yu, Jianfeng & Yuan, Yu, 2012. "The short of it: Investor sentiment and anomalies," Journal of Financial Economics, Elsevier, vol. 104(2), pages 288-302.
    28. Lutz Kilian & Robert J. Vigfusson, 2017. "The Role of Oil Price Shocks in Causing U.S. Recessions," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 49(8), pages 1747-1776, December.
    29. Elliott, Graham & Gargano, Antonio & Timmermann, Allan, 2013. "Complete subset regressions," Journal of Econometrics, Elsevier, vol. 177(2), pages 357-373.
    30. Ludvigson, Sydney C. & Ng, Serena, 2007. "The empirical risk-return relation: A factor analysis approach," Journal of Financial Economics, Elsevier, vol. 83(1), pages 171-222, January.
    31. Stock, James H. & Watson, Mark W., 2006. "Forecasting with Many Predictors," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 10, pages 515-554, Elsevier.
    32. Narayan, Paresh Kumar & Liu, Ruipeng, 2018. "A new GARCH model with higher moments for stock return predictability," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 56(C), pages 93-103.
    33. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    34. Hou, Keqiang & Mountain, Dean C. & Wu, Ting, 2016. "Oil price shocks and their transmission mechanism in an oil-exporting economy: A VAR analysis informed by a DSGE model," Journal of International Money and Finance, Elsevier, vol. 68(C), pages 21-49.
    35. Lutz Kilian, 2009. "Not All Oil Price Shocks Are Alike: Disentangling Demand and Supply Shocks in the Crude Oil Market," American Economic Review, American Economic Association, vol. 99(3), pages 1053-1069, June.
    36. Barbara Rossi & Atsushi Inoue, 2012. "Out-of-Sample Forecast Tests Robust to the Choice of Window Size," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(3), pages 432-453, April.
    37. Christopher J. Neely & David E. Rapach & Jun Tu & Guofu Zhou, 2014. "Forecasting the Equity Risk Premium: The Role of Technical Indicators," Management Science, INFORMS, vol. 60(7), pages 1772-1791, July.
    38. Baumeister, Christiane & Kilian, Lutz & Lee, Thomas K., 2014. "Are there gains from pooling real-time oil price forecasts?," Energy Economics, Elsevier, vol. 46(S1), pages 33-43.
    39. Çakmaklı, Cem & van Dijk, Dick, 2016. "Getting the most out of macroeconomic information for predicting excess stock returns," International Journal of Forecasting, Elsevier, vol. 32(3), pages 650-668.
    40. Zhang, Yaojie & Zeng, Qing & Ma, Feng & Shi, Benshan, 2019. "Forecasting stock returns: Do less powerful predictors help?," Economic Modelling, Elsevier, vol. 78(C), pages 32-39.
    41. Kelly, Bryan & Pruitt, Seth, 2015. "The three-pass regression filter: A new approach to forecasting using many predictors," Journal of Econometrics, Elsevier, vol. 186(2), pages 294-316.
    42. Robert Tibshirani & Michael Saunders & Saharon Rosset & Ji Zhu & Keith Knight, 2005. "Sparsity and smoothness via the fused lasso," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(1), pages 91-108, February.
    43. Jiang, Fuwei & Lee, Joshua & Martin, Xiumin & Zhou, Guofu, 2019. "Manager sentiment and stock returns," Journal of Financial Economics, Elsevier, vol. 132(1), pages 126-149.
    44. Bryan Kelly & Seth Pruitt, 2013. "Market Expectations in the Cross-Section of Present Values," Journal of Finance, American Finance Association, vol. 68(5), pages 1721-1756, October.
    45. Sydney C. Ludvigson & Serena Ng, 2009. "Macro Factors in Bond Risk Premia," Review of Financial Studies, Society for Financial Studies, vol. 22(12), pages 5027-5067, December.
    46. John Y. Campbell & Samuel B. Thompson, 2008. "Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?," Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1509-1531, July.
    47. Yin, Libo & Yang, Qingyuan, 2016. "Predicting the oil prices: Do technical indicators help?," Energy Economics, Elsevier, vol. 56(C), pages 338-350.
    48. Harvey, David I & Leybourne, Stephen J & Newbold, Paul, 1998. "Tests for Forecast Encompassing," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(2), pages 254-259, April.
    49. Rapach, David E. & Ringgenberg, Matthew C. & Zhou, Guofu, 2016. "Short interest and aggregate stock returns," Journal of Financial Economics, Elsevier, vol. 121(1), pages 46-65.
    50. Wang, Yudong & Liu, Li & Ma, Feng & Diao, Xundi, 2018. "Momentum of return predictability," Journal of Empirical Finance, Elsevier, vol. 45(C), pages 141-156.
    51. Hai Lin & Chunchi Wu & Guofu Zhou, 2018. "Forecasting Corporate Bond Returns with a Large Set of Predictors: An Iterated Combination Approach," Management Science, INFORMS, vol. 64(9), pages 4218-4238, September.
    52. Naser, Hanan, 2016. "Estimating and forecasting the real prices of crude oil: A data rich model using a dynamic model averaging (DMA) approach," Energy Economics, Elsevier, vol. 56(C), pages 75-87.
    53. Andrew B. Abel & Janice C. Eberly & Stavros Panageas, 2013. "Optimal Inattention to the Stock Market With Information Costs and Transactions Costs," Econometrica, Econometric Society, vol. 81(4), pages 1455-1481, July.
    54. Han, Liyan & Lv, Qiuna & Yin, Libo, 2017. "Can investor attention predict oil prices?," Energy Economics, Elsevier, vol. 66(C), pages 547-558.
    55. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    56. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    57. David E. Rapach & Jack K. Strauss & Guofu Zhou, 2010. "Out-of-Sample Equity Premium Prediction: Combination Forecasts and Links to the Real Economy," Review of Financial Studies, Society for Financial Studies, vol. 23(2), pages 821-862, February.
    58. G. Elliott & C. Granger & A. Timmermann (ed.), 2013. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 2, number 2.
    59. Zhang, Yaojie & Ma, Feng & Shi, Benshan & Huang, Dengshi, 2018. "Forecasting the prices of crude oil: An iterated combination approach," Energy Economics, Elsevier, vol. 70(C), pages 472-483.
    60. Yu, Jianfeng & Yuan, Yu, 2011. "Investor sentiment and the mean-variance relation," Journal of Financial Economics, Elsevier, vol. 100(2), pages 367-381, May.
    61. Leitch, Gordon & Tanner, J Ernest, 1991. "Economic Forecast Evaluation: Profits versus the Conventional Error Measures," American Economic Review, American Economic Association, vol. 81(3), pages 580-590, June.
    62. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    63. repec:zbw:bofrdp:2016_029 is not listed on IDEAS
    64. Lin, Qi, 2018. "Technical analysis and stock return predictability: An aligned approach," Journal of Financial Markets, Elsevier, vol. 38(C), pages 103-123.
    65. Ming Yuan & Yi Lin, 2006. "Model selection and estimation in regression with grouped variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(1), pages 49-67, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. He, Mengxi & Zhang, Yaojie & Wen, Danyan & Wang, Yudong, 2021. "Forecasting crude oil prices: A scaled PCA approach," Energy Economics, Elsevier, vol. 97(C).
    2. Zhang, Yaojie & Wang, Yudong, 2023. "Forecasting crude oil futures market returns: A principal component analysis combination approach," International Journal of Forecasting, Elsevier, vol. 39(2), pages 659-673.
    3. Zhang, Yaojie & Ma, Feng & Shi, Benshan & Huang, Dengshi, 2018. "Forecasting the prices of crude oil: An iterated combination approach," Energy Economics, Elsevier, vol. 70(C), pages 472-483.
    4. Xing, Li-Min & Zhang, Yue-Jun, 2022. "Forecasting crude oil prices with shrinkage methods: Can nonconvex penalty and Huber loss help?," Energy Economics, Elsevier, vol. 110(C).
    5. Dai, Zhifeng & Kang, Jie, 2021. "Bond yield and crude oil prices predictability," Energy Economics, Elsevier, vol. 97(C).
    6. Wen, Danyan & Liu, Li & Wang, Yudong & Zhang, Yaojie, 2022. "Forecasting crude oil market returns: Enhanced moving average technical indicators," Resources Policy, Elsevier, vol. 76(C).
    7. Dai, Zhifeng & Kang, Jie & Hu, Yangli, 2021. "Efficient predictability of oil price: The role of number of IPOs and U.S. dollar index," Resources Policy, Elsevier, vol. 74(C).
    8. Yaojie Zhang & Yudong Wang & Feng Ma, 2021. "Forecasting US stock market volatility: How to use international volatility information," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(5), pages 733-768, August.
    9. Zhang, Yaojie & Wei, Yu & Zhang, Yi & Jin, Daxiang, 2019. "Forecasting oil price volatility: Forecast combination versus shrinkage method," Energy Economics, Elsevier, vol. 80(C), pages 423-433.
    10. Tian, Guangning & Peng, Yuchao & Meng, Yuhao, 2023. "Forecasting crude oil prices in the COVID-19 era: Can machine learn better?," Energy Economics, Elsevier, vol. 125(C).
    11. Zhang, Yaojie & Wahab, M.I.M. & Wang, Yudong, 2023. "Forecasting crude oil market volatility using variable selection and common factor," International Journal of Forecasting, Elsevier, vol. 39(1), pages 486-502.
    12. Zhang, Zhikai & He, Mengxi & Zhang, Yaojie & Wang, Yudong, 2022. "Geopolitical risk trends and crude oil price predictability," Energy, Elsevier, vol. 258(C).
    13. Dai, Zhifeng & Zhu, Huan, 2020. "Stock return predictability from a mixed model perspective," Pacific-Basin Finance Journal, Elsevier, vol. 60(C).
    14. Hao, Xianfeng & Zhao, Yuyang & Wang, Yudong, 2020. "Forecasting the real prices of crude oil using robust regression models with regularization constraints," Energy Economics, Elsevier, vol. 86(C).
    15. Dai, Zhifeng & Zhu, Huan, 2021. "Indicator selection and stock return predictability," The North American Journal of Economics and Finance, Elsevier, vol. 57(C).
    16. Zhang, Yaojie & Ma, Feng & Zhu, Bo, 2019. "Intraday momentum and stock return predictability: Evidence from China," Economic Modelling, Elsevier, vol. 76(C), pages 319-329.
    17. Wen, Chufu & Zhu, Haoyang & Dai, Zhifeng, 2023. "Forecasting commodity prices returns: The role of partial least squares approach," Energy Economics, Elsevier, vol. 125(C).
    18. Zhang, Yaojie & Zeng, Qing & Ma, Feng & Shi, Benshan, 2019. "Forecasting stock returns: Do less powerful predictors help?," Economic Modelling, Elsevier, vol. 78(C), pages 32-39.
    19. Yaojie Zhang & Feng Ma & Chao Liang & Yi Zhang, 2021. "Good variance, bad variance, and stock return predictability," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 4410-4423, July.
    20. Zhifeng Dai & Jie Kang & Hua Yin, 2023. "Forecasting equity risk premium: A new method based on wavelet de‐noising," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 28(4), pages 4331-4352, October.

    More about this item

    Keywords

    Oil price predictability; Out-of-sample forecasts; Lasso; Elastic net; Variable selection;
    All these keywords.

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • Q47 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Energy - - - Energy Forecasting
    • G17 - Financial Economics - - General Financial Markets - - - Financial Forecasting and Simulation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:empfin:v:54:y:2019:i:c:p:97-117. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jempfin .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.