IDEAS home Printed from https://ideas.repec.org/a/spr/annopr/v348y2025i1d10.1007_s10479-023-05546-5.html
   My bibliography  Save this article

Forecasting portfolio variance: a new decomposition approach

Author

Listed:
  • Bo Yu

    (Southwestern University of Finance and Economics)

  • Dayong Zhang

    (Southwestern University of Finance and Economics)

  • Qiang Ji

    (Chinese Academy of Sciences)

Abstract

This paper proposes a new decomposition approach by separating realized covariation into components based on signs (positive and negative) and magnitudes (continuous, small jump, and large jump). The motivation behind this decomposition is that certain variation components can be more useful in forecasting than others. Including only “information-rich” components in realized (co)-variation forecasting models can improve predictive accuracy. Using various machine learning models, the marginal predictive content of each variation components can be assessed. The empirical exercise is based on all constituent of S &P 500 stocks between 2010 and 2019. We find that standard machine learning methods without the more granular variation measures offer limited improvement to out-of-sample fit (likely due to low signal-to-noise ratios) relative to benchmark HAR-type forecasting models. However, sparse models, which are specified using predictors selected using a first “variable selection” yield significant improvements in predictive accuracy when the decomposed variation measures are included. These predictive gains can be traced to the identification of short-lived pricing signals associated with co-jumps.

Suggested Citation

  • Bo Yu & Dayong Zhang & Qiang Ji, 2025. "Forecasting portfolio variance: a new decomposition approach," Annals of Operations Research, Springer, vol. 348(1), pages 543-578, May.
  • Handle: RePEc:spr:annopr:v:348:y:2025:i:1:d:10.1007_s10479-023-05546-5
    DOI: 10.1007/s10479-023-05546-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10479-023-05546-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10479-023-05546-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Kelly, Bryan T. & Pruitt, Seth & Su, Yinan, 2019. "Characteristics are covariances: A unified model of risk and return," Journal of Financial Economics, Elsevier, vol. 134(3), pages 501-524.
    2. Corsi, Fulvio & Pirino, Davide & Renò, Roberto, 2010. "Threshold bipower variation and the impact of jumps on volatility forecasting," Journal of Econometrics, Elsevier, vol. 159(2), pages 276-288, December.
    3. Guanhao Feng & Stefano Giglio & Dacheng Xiu, 2020. "Taming the Factor Zoo: A Test of New Factors," Journal of Finance, American Finance Association, vol. 75(3), pages 1327-1370, June.
    4. Xi Dong & Yan Li & David E. Rapach & Guofu Zhou, 2022. "Anomalies and the Expected Market Return," Journal of Finance, American Finance Association, vol. 77(1), pages 639-681, February.
    5. Louis K.C. Chan & Jason Karceski & Josef Lakonishok, 1999. "On Portfolio Optimization: Forecasting Covariances and Choosing the Risk Model," NBER Working Papers 7039, National Bureau of Economic Research, Inc.
    6. Yacine Aït-Sahalia & Jean Jacod, 2012. "Analyzing the Spectrum of Asset Returns: Jump and Volatility Components in High Frequency Data," Journal of Economic Literature, American Economic Association, vol. 50(4), pages 1007-1050, December.
    7. A. Belloni & D. Chen & V. Chernozhukov & C. Hansen, 2012. "Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain," Econometrica, Econometric Society, vol. 80(6), pages 2369-2429, November.
    8. Amaya, Diego & Christoffersen, Peter & Jacobs, Kris & Vasquez, Aurelio, 2015. "Does realized skewness predict the cross-section of equity returns?," Journal of Financial Economics, Elsevier, vol. 118(1), pages 135-167.
    9. Stock, James H. & Watson, Mark W., 2006. "Forecasting with Many Predictors," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 10, pages 515-554, Elsevier.
    10. Fulvio Corsi, 2009. "A Simple Approximate Long-Memory Model of Realized Volatility," Journal of Financial Econometrics, Oxford University Press, vol. 7(2), pages 174-196, Spring.
    11. Bollerslev, Tim & Todorov, Viktor & Li, Sophia Zhengzi, 2013. "Jump tails, extreme dependencies, and the distribution of stock returns," Journal of Econometrics, Elsevier, vol. 172(2), pages 307-324.
    12. Bryan Kelly & Seth Pruitt, 2013. "Market Expectations in the Cross-Section of Present Values," Journal of Finance, American Finance Association, vol. 68(5), pages 1721-1756, October.
    13. Kozak, Serhiy & Nagel, Stefan & Santosh, Shrihari, 2020. "Shrinking the cross-section," Journal of Financial Economics, Elsevier, vol. 135(2), pages 271-292.
    14. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    15. Xavier Gabaix, 2014. "A Sparsity-Based Model of Bounded Rationality," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(4), pages 1661-1710.
    16. Bruno Feunou & Mohammad R Jahan-Parvar & Cédric Okou, 2018. "Downside Variance Risk Premium," Journal of Financial Econometrics, Oxford University Press, vol. 16(3), pages 341-383.
    17. Cecilia Mancini, 2009. "Non‐parametric Threshold Estimation for Models with Stochastic Diffusion Coefficient and Jumps," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 36(2), pages 270-296, June.
    18. David E. Rapach & Jack K. Strauss & Guofu Zhou, 2013. "International Stock Return Predictability: What Is the Role of the United States?," Journal of Finance, American Finance Association, vol. 68(4), pages 1633-1662, August.
    19. John Y. Campbell & Samuel B. Thompson, 2008. "Predicting Excess Stock Returns Out of Sample: Can Anything Beat the Historical Average?," The Review of Financial Studies, Society for Financial Studies, vol. 21(4), pages 1509-1531, July.
    20. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.
    21. Barndorff-Nielsen, Ole E. & Graversen, Svend Erik & Jacod, Jean & Shephard, Neil, 2006. "Limit Theorems For Bipower Variation In Financial Econometrics," Econometric Theory, Cambridge University Press, vol. 22(4), pages 677-719, August.
    22. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    23. Tim Bollerslev & Jia Li & Andrew J. Patton & Rogier Quaedvlieg, 2020. "Realized Semicovariances," Econometrica, Econometric Society, vol. 88(4), pages 1515-1551, July.
    24. Andrew J. Patton & Kevin Sheppard, 2015. "Good Volatility, Bad Volatility: Signed Jumps and The Persistence of Volatility," The Review of Economics and Statistics, MIT Press, vol. 97(3), pages 683-697, July.
    25. Li, Jia & Todorov, Viktor & Tauchen, George & Chen, Rui, 2017. "Mixed-scale jump regressions with bootstrap inference," Journal of Econometrics, Elsevier, vol. 201(2), pages 417-432.
    26. repec:hal:journl:peer-00741630 is not listed on IDEAS
    27. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    28. Kelly, Bryan & Pruitt, Seth, 2015. "The three-pass regression filter: A new approach to forecasting using many predictors," Journal of Econometrics, Elsevier, vol. 186(2), pages 294-316.
    29. Alex Chinco & Adam D. Clark‐Joseph & Mao Ye, 2019. "Sparse Signals in the Cross‐Section of Returns," Journal of Finance, American Finance Association, vol. 74(1), pages 449-492, February.
    30. Becker, R. & Clements, A.E. & Doolan, M.B. & Hurn, A.S., 2015. "Selecting volatility forecasting models for portfolio allocation purposes," International Journal of Forecasting, Elsevier, vol. 31(3), pages 849-861.
    31. Stefano Giglio & Dacheng Xiu, 2021. "Asset Pricing with Omitted Factors," Journal of Political Economy, University of Chicago Press, vol. 129(7), pages 1947-1990.
    32. Bollerslev, Tim & Li, Sophia Zhengzi & Zhao, Bingzhi, 2020. "Good Volatility, Bad Volatility, and the Cross Section of Stock Returns," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 55(3), pages 751-781, May.
    33. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    34. Duong, Diep & Swanson, Norman R., 2015. "Empirical evidence on the importance of aggregation, asymmetry, and jumps for volatility prediction," Journal of Econometrics, Elsevier, vol. 187(2), pages 606-621.
    35. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    36. Chan, Louis K C & Karceski, Jason & Lakonishok, Josef, 1999. "On Portfolio Optimization: Forecasting Covariances and Choosing the Risk Model," The Review of Financial Studies, Society for Financial Studies, vol. 12(5), pages 937-974.
    37. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    38. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    39. Bo Yu & Bruce Mizrach & Norman R. Swanson, 2020. "New Evidence of the Marginal Predictive Content of Small and Large Jumps in the Cross-Section," Econometrics, MDPI, vol. 8(2), pages 1-52, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cheng, Mingmian & Swanson, Norman R. & Yang, Xiye, 2021. "Forecasting volatility using double shrinkage methods," Journal of Empirical Finance, Elsevier, vol. 62(C), pages 46-61.
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Vigo Pereira, Caio, 2021. "Portfolio efficiency with high-dimensional data as conditioning information," International Review of Financial Analysis, Elsevier, vol. 77(C).
    4. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    5. Thomas Conlon & John Cotter & Iason Kynigakis, 2021. "Machine Learning and Factor-Based Portfolio Optimization," Papers 2107.13866, arXiv.org.
    6. Bo Yu & Bruce Mizrach & Norman R. Swanson, 2020. "New Evidence of the Marginal Predictive Content of Small and Large Jumps in the Cross-Section," Econometrics, MDPI, vol. 8(2), pages 1-52, May.
    7. Li, Zhao-Chen & Xie, Chi & Wang, Gang-Jin & Zhu, You & Zeng, Zhi-Jian & Gong, Jue, 2024. "Forecasting global stock market volatilities: A shrinkage heterogeneous autoregressive (HAR) model with a large cross-market predictor set," International Review of Economics & Finance, Elsevier, vol. 93(PB), pages 673-711.
    8. Zhang, Hongwei & Zhao, Xinyi & Gao, Wang & Niu, Zibo, 2023. "The role of higher moments in predicting China's oil futures volatility: Evidence from machine learning models," Journal of Commodity Markets, Elsevier, vol. 32(C).
    9. Hoang, Daniel & Wiegratz, Kevin, 2022. "Machine learning methods in finance: Recent applications and prospects," Working Paper Series in Economics 158, Karlsruhe Institute of Technology (KIT), Department of Economics and Management.
    10. Mekelburg, Erik & Strauss, Jack, 2024. "Pooling and winsorizing machine learning forecasts to predict stock returns with high-dimensional data," Journal of Empirical Finance, Elsevier, vol. 79(C).
    11. Niu, Zibo & Demirer, Riza & Suleman, Muhammad Tahir & Zhang, Hongwei & Zhu, Xuehong, 2024. "Do industries predict stock market volatility? Evidence from machine learning models," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 90(C).
    12. Bennett, Donyetta & Mekelburg, Erik & Strauss, Jack & Williams, T.H., 2024. "Unlocking the black box of sentiment and cryptocurrency: What, which, why, when and how?," Global Finance Journal, Elsevier, vol. 60(C).
    13. Cheng, Mingmian & Liao, Yuan & Yang, Xiye, 2023. "Uniform predictive inference for factor models with instrumental and idiosyncratic betas," Journal of Econometrics, Elsevier, vol. 237(2).
    14. Nusret Cakici & Christian Fieberg & Daniel Metko & Adam Zaremba, 2024. "Do Anomalies Really Predict Market Returns? New Data and New Evidence," Review of Finance, European Finance Association, vol. 28(1), pages 1-44.
    15. Mykola Babiak & Jozef Barunik, 2020. "Deep Learning, Predictability, and Optimal Portfolio Returns," CERGE-EI Working Papers wp677, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    16. Chun, Dohyun & Cho, Hoon & Ryu, Doojin, 2025. "Volatility forecasting and volatility-timing strategies: A machine learning approach," Research in International Business and Finance, Elsevier, vol. 75(C).
    17. Xi Dong & Yan Li & David E. Rapach & Guofu Zhou, 2022. "Anomalies and the Expected Market Return," Journal of Finance, American Finance Association, vol. 77(1), pages 639-681, February.
    18. Wen, Danyan & He, Mengxi & Wang, Yudong & Zhang, Yaojie, 2024. "Forecasting crude oil market volatility: A comprehensive look at uncertainty variables," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1022-1041.
    19. Bae, Juhee, 2024. "Factor-augmented forecasting in big data," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1660-1688.
    20. Huang, Dashan & Li, Jiangyuan & Wang, Liyao, 2021. "Are disagreements agreeable? Evidence from information aggregation," Journal of Financial Economics, Elsevier, vol. 141(1), pages 83-101.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C58 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Financial Econometrics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:annopr:v:348:y:2025:i:1:d:10.1007_s10479-023-05546-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.