IDEAS home Printed from https://ideas.repec.org/p/fip/fedgfe/103519.html

Capturing Heterogeneity: Machine Learning Approaches to Implied Volatility Forecasting

Author

Abstract

Despite documented heterogeneity in volatility dynamics across the option surface, standard implied volatility forecasting models apply homogeneous parameters throughout. We introduce a machine-learning framework that uses regression trees to partition the surface along both moneyness and maturity dimensions, identifying data-driven regions where distinct forecasting models perform best. Extending the Surface Heterogeneous Autoregressive (SHAR) framework of Dufays, Jacobs, and Rombouts (2025), we develop tree-based SHAR specifications that preserve interpretable structure while allowing model parameters to vary across the surface. Empirical analysis using S&P 500 options demonstrates that the boosted tree-based specification achieves the lowest out-of-sample forecast errors across all horizons, reducing one-month-ahead RMSE by 13 percent versus the benchmark SHAR model. The improvements are statistically significant and particularly pronounced during stress periods. The estimated tree presents economically interpretable segmentation: short-dated options exhibit higher daily persistence but lower monthly persistence than long-dated options, while deep out-of-the-money calls or puts display distinct dynamics from near-the-money contracts.

Suggested Citation

  • Hyung Joo Kim & Dong Hwan Oh, 2026. "Capturing Heterogeneity: Machine Learning Approaches to Implied Volatility Forecasting," Finance and Economics Discussion Series 2026-049, Board of Governors of the Federal Reserve System (U.S.).
  • Handle: RePEc:fip:fedgfe:103519
    DOI: 10.17016/FEDS.2026.049
    as

    Download full text from publisher

    File URL: https://www.federalreserve.gov/econres/feds/files/2026049pap.pdf
    File Function: Full text
    Download Restriction: no

    File URL: https://libkey.io/10.17016/FEDS.2026.049?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Eaton, Gregory W. & Green, T. Clifton & Roseman, Brian S. & Wu, Yanbin, 2026. "Retail option traders and the implied volatility surface," Journal of Financial Economics, Elsevier, vol. 177(C).
    2. Pan, Jun, 2002. "The jump-risk premia implicit in options: evidence from an integrated time-series study," Journal of Financial Economics, Elsevier, vol. 63(1), pages 3-50, January.
    3. A. S. Hurn & K. A. Lindsay & A. J. McClelland, 2015. "Estimating the Parameters of Stochastic Volatility Models Using Option Price Data," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(4), pages 579-594, October.
    4. Bakshi, Gurdip & Cao, Charles & Chen, Zhiwu, 1997. "Empirical Performance of Alternative Option Pricing Models," Journal of Finance, American Finance Association, vol. 52(5), pages 2003-2049, December.
    5. Bakshi, Gurdip & Madan, Dilip & Panayotov, George, 2010. "Returns of claims on the upside and the viability of U-shaped pricing kernels," Journal of Financial Economics, Elsevier, vol. 97(1), pages 130-154, July.
    6. Peter Christoffersen & Steven Heston & Kris Jacobs, 2009. "The Shape and Term Structure of the Index Option Smirk: Why Multifactor Stochastic Volatility Models Work So Well," Management Science, INFORMS, vol. 55(12), pages 1914-1932, December.
    7. Peter R. Hansen & Asger Lunde & James M. Nason, 2011. "The Model Confidence Set," Econometrica, Econometric Society, vol. 79(2), pages 453-497, March.
    8. Heston, Steven L, 1993. "A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options," The Review of Financial Studies, Society for Financial Studies, vol. 6(2), pages 327-343.
    9. Oh, Dong Hwan & Patton, Andrew J., 2024. "Better the devil you know: Improved forecasts from imperfect models," Journal of Econometrics, Elsevier, vol. 242(1).
    10. Fulvio Corsi, 2009. "A Simple Approximate Long-Memory Model of Realized Volatility," Journal of Financial Econometrics, Oxford University Press, vol. 7(2), pages 174-196, Spring.
    11. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    12. J. Fan & M. Farmen & I. Gijbels, 1998. "Local maximum likelihood estimation and inference," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(3), pages 591-608.
    13. repec:bla:jfinan:v:59:y:2004:i:2:p:711-753 is not listed on IDEAS
    14. Fan, Jianqing & Mancini, Loriano, 2009. "Option Pricing With Model-Guided Nonparametric Methods," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1351-1372.
    15. Nicolae Garleanu & Lasse Heje Pedersen & Allen M. Poteshman, 2009. "Demand-Based Option Pricing," The Review of Financial Studies, Society for Financial Studies, vol. 22(10), pages 4259-4299, October.
    16. Andrea Buraschi & Alexei Jiltsov, 2006. "Model Uncertainty and Option Markets with Heterogeneous Beliefs," Journal of Finance, American Finance Association, vol. 61(6), pages 2841-2897, December.
    17. Hyung Joo Kim & Dong Hwan Oh, 2025. "Local Estimation for Option Pricing: Improving Forecasts with Market State Information," Finance and Economics Discussion Series 2025-076, Board of Governors of the Federal Reserve System (U.S.).
    18. Black, Fischer & Scholes, Myron S, 1973. "The Pricing of Options and Corporate Liabilities," Journal of Political Economy, University of Chicago Press, vol. 81(3), pages 637-654, May-June.
    19. repec:bla:jfinan:v:53:y:1998:i:6:p:2059-2106 is not listed on IDEAS
    20. Schreindorfer, David & Sichert, Tobias, 2025. "Conditional risk and the pricing kernel," Journal of Financial Economics, Elsevier, vol. 171(C).
    21. Philippe Goulet Coulombe, 2024. "The macroeconomy as a random forest," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(3), pages 401-421, April.
    22. Tyler Beason & David Schreindorfer, 2022. "Dissecting the Equity Premium," Journal of Political Economy, University of Chicago Press, vol. 130(8), pages 2203-2222.
    23. Marcelo C. Medeiros & Gabriel F. R. Vasconcelos & Álvaro Veiga & Eduardo Zilberman, 2021. "Forecasting Inflation in a Data-Rich Environment: The Benefits of Machine Learning Methods," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 98-119, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hyung Joo Kim & Dong Hwan Oh, 2025. "Local Estimation for Option Pricing: Improving Forecasts with Market State Information," Finance and Economics Discussion Series 2025-076, Board of Governors of the Federal Reserve System (U.S.).
    2. Kaeck, Andreas & Seeger, Norman J., 2020. "VIX derivatives, hedging and vol-of-vol risk," European Journal of Operational Research, Elsevier, vol. 283(2), pages 767-782.
    3. Christoffersen, Peter & Heston, Steven & Jacobs, Kris, 2010. "Option Anomalies and the Pricing Kernel," Working Papers 11-17, University of Pennsylvania, Wharton School, Weiss Center.
    4. Augustyniak, Maciej & Badescu, Alexandru & Bégin, Jean-François, 2023. "A discrete-time hedging framework with multiple factors and fat tails: On what matters," Journal of Econometrics, Elsevier, vol. 232(2), pages 416-444.
    5. Yu, Junhong & Ruan, Xinfeng & Fan, Zheqi, 2025. "Merton (1976) implied jump," Journal of Economic Dynamics and Control, Elsevier, vol. 180(C).
    6. Liu, Xiaoquan & Cao, Yi & Ma, Chenghu & Shen, Liya, 2019. "Wavelet-based option pricing: An empirical study," European Journal of Operational Research, Elsevier, vol. 272(3), pages 1132-1142.
    7. Kadir G. Babaoglou & Peter Christoffersen & Steven L. Heston & Kris Jacobs, 2014. "Option Valuation with Volatility Components, Fat Tails, and Nonlinear Pricing Kernels," CREATES Research Papers 2015-55, Department of Economics and Business Economics, Aarhus University.
    8. Wu, Bin & Chen, Pengzhan & Ye, Wuyi, 2024. "Variance swaps with mean reversion and multi-factor variance," European Journal of Operational Research, Elsevier, vol. 315(1), pages 191-212.
    9. Diego Amaya & Jean-François Bégin & Geneviève Gauthier, 2022. "The Informational Content of High-Frequency Option Prices," Management Science, INFORMS, vol. 68(3), pages 2166-2201, March.
    10. Nikola Gradojevic & Dragan Kukolj, 2024. "Unlocking the black box: Non-parametric option pricing before and during COVID-19," Annals of Operations Research, Springer, vol. 334(1), pages 59-82, March.
    11. Maung, Kenwin & Swanson, Norman R., 2025. "A survey of models and methods used for forecasting when investing in financial markets," International Journal of Forecasting, Elsevier, vol. 41(4), pages 1355-1382.
    12. Alexander David & Pietro Veronesi, 2014. "Investors' and Central Bank's Uncertainty Embedded in Index Options," The Review of Financial Studies, Society for Financial Studies, vol. 27(6), pages 1661-1716.
    13. Steven Heston & Kris Jacobs & Hyung Joo Kim, 2023. "The Pricing Kernel in Options," Finance and Economics Discussion Series 2023-053, Board of Governors of the Federal Reserve System (U.S.).
    14. Audrino, Francesco & Fengler, Matthias R., 2015. "Are classical option pricing models consistent with observed option second-order moments? Evidence from high-frequency data," Journal of Banking & Finance, Elsevier, vol. 61(C), pages 46-63.
    15. Ghanbari, Hamed, 2024. "Persistent and transient variance components in option pricing models with variance-dependent Kernel," Journal of Empirical Finance, Elsevier, vol. 79(C).
    16. Benzoni, Luca & Collin-Dufresne, Pierre & Goldstein, Robert S., 2011. "Explaining asset pricing puzzles associated with the 1987 market crash," Journal of Financial Economics, Elsevier, vol. 101(3), pages 552-573, September.
    17. Bégin, Jean-François & Gómez, Fabio & Ignatieva, Katja & Li, Han, 2025. "The stochastic behavior of electricity prices under scrutiny: Evidence from spot and futures markets," Energy Economics, Elsevier, vol. 144(C).
    18. Constantinides, George M. & Lian, Lei, 2021. "The Supply and Demand of S&P 500 Put Options," Critical Finance Review, now publishers, vol. 10(1), pages 1-20, April.
    19. Stylianos Perrakis, 2022. "From innovation to obfuscation: continuous time finance fifty years later," Financial Markets and Portfolio Management, Springer;Swiss Society for Financial Market Research, vol. 36(3), pages 369-401, September.
    20. George M. Constantinides & Michal Czerwonko & Stylianos Perrakis, 2020. "Mispriced index option portfolios," Financial Management, Financial Management Association International, vol. 49(2), pages 297-330, June.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C58 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Financial Econometrics
    • G12 - Financial Economics - - General Financial Markets - - - Asset Pricing; Trading Volume; Bond Interest Rates

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedgfe:103519. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ryan Wolfslayer ; Keisha Fournillier (email available below). General contact details of provider: https://edirc.repec.org/data/frbgvus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.