IDEAS home Printed from https://ideas.repec.org/a/inm/orijds/v4y2025i2p114-132.html

Nonstationary and Sparsely-Correlated Multioutput Gaussian Process with Spike-and-Slab Prior

Author

Listed:
  • Xinming Wang

    (Department of Industrial Engineering and Management, Peking University, Beijing 100871, China)

  • Yongxiang Li

    (Department of Industrial Engineering and Management, Shanghai Jiaotong University, Shanghai 200240, China)

  • Xiaowei Yue

    (Department of Industrial Engineering, Tsinghua University, Beijing 100084, China)

  • Jianguo Wu

    (Department of Industrial Engineering and Management, Peking University, Beijing 100871, China)

Abstract

Multioutput Gaussian process (MGP) is commonly used as a transfer learning method to leverage information among multiple outputs. A key advantage of MGP is providing uncertainty quantification for prediction, which is highly important for subsequent decision-making tasks. However, traditional MGP may not be sufficiently flexible to handle multivariate data with dynamic characteristics, particularly when dealing with complex temporal correlations. Additionally, because some outputs may lack correlation, transferring information among them may lead to negative transfer. To address these issues, this study proposes a nonstationary MGP model that can capture both the dynamic and sparse correlation among outputs. Specifically, the covariance functions of MGP are constructed using convolutions of time-varying kernel functions. Then a dynamic spike-and-slab prior is placed on correlation parameters to automatically decide which sources are informative to the target output in the training process. An expectation-maximization (EM) algorithm is proposed for efficient model fitting. Both numerical studies and real cases demonstrate its efficacy in capturing dynamic and sparse correlation structure and mitigating negative transfer for high-dimensional time-series data modeling. A mountain-car reinforcement learning case highlights that transferring knowledge from source expertise can enhance the efficiency of the target decision-making process. Our method holds promise for application in more complex multitask decision-making challenges within nonstationary environments in the future. History: Rema Padman served as the senior editor for this article. Funding: This work was supported by NSFC [Grants NSFC-72171003, NSFC-71932006, and NSFC-72101147]. Data Ethics & Reproducibility Note: The code capsule is available on Code Ocean at https://doi.org/10.24433/CO.4010696.v1 and in the e-Companion to this article (available at https://doi.org/10.1287/ijds.2023.0022 ).

Suggested Citation

  • Xinming Wang & Yongxiang Li & Xiaowei Yue & Jianguo Wu, 2025. "Nonstationary and Sparsely-Correlated Multioutput Gaussian Process with Spike-and-Slab Prior," INFORMS Joural on Data Science, INFORMS, vol. 4(2), pages 114-132, April.
    • Handle: RePEc:inm:orijds:v:4:y:2025:i:2:p:114-132
    DOI: 10.1287/ijds.2023.0022
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/ijds.2023.0022
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijds.2023.0022?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Veronika Ročková & Edward I. George, 2014. "EMVS: The EM Approach to Bayesian Variable Selection," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(506), pages 828-846, June.
    2. Jongwoo Ko & Heeyoung Kim, 2022. "Deep Gaussian process models for integrating multifidelity experiments with nonstationary relationships," IISE Transactions, Taylor & Francis Journals, vol. 54(7), pages 686-698, July.
    3. Linmiao Zhang & Kaibo Wang & Nan Chen, 2016. "Monitoring wafers’ geometric quality using an additive Gaussian process model," IISE Transactions, Taylor & Francis Journals, vol. 48(1), pages 1-15, January.
    4. Alan Gelfand & Alexandra Schmidt & Sudipto Banerjee & C. Sirmans, 2004. "Nonstationary multivariate process modeling through spatially varying coregionalization," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 13(2), pages 263-312, December.
    5. Sooin Yun & Xianyang Zhang & Bo Li, 2022. "Detection of Local Differences in Spatial Characteristics Between Two Spatiotemporal Random Fields," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 117(537), pages 291-306, January.
    6. Florian Huber & Gary Koop & Luca Onorante, 2021. "Inducing Sparsity and Shrinkage in Time-Varying Parameter Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(3), pages 669-683, July.
    7. Kalli, Maria & Griffin, Jim E., 2014. "Time-varying sparsity in dynamic regression models," Journal of Econometrics, Elsevier, vol. 178(2), pages 779-793.
    8. Lyndsay Shand & Bo Li, 2017. "Modeling nonstationarity in space and time," Biometrics, The International Biometric Society, vol. 73(3), pages 759-768, September.
    9. Gramacy, Robert B & Lee, Herbert K. H, 2008. "Bayesian Treed Gaussian Process Models With an Application to Computer Modeling," Journal of the American Statistical Association, American Statistical Association, vol. 103(483), pages 1119-1130.
    10. Bo Shen & Raghav Gnanasambandam & Rongxuan Wang & Zhenyu James Kong, 2023. "Multi-task Gaussian process upper confidence bound for hyperparameter tuning and its application for simulation studies of additive manufacturing," IISE Transactions, Taylor & Francis Journals, vol. 55(5), pages 496-508, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Florian Huber & Luca Rossini, 2020. "Inference in Bayesian Additive Vector Autoregressive Tree Models," Papers 2006.16333, arXiv.org, revised Mar 2021.
    2. Dufays, Arnaud & Rombouts, Jeroen V.K., 2020. "Relevant parameter changes in structural break models," Journal of Econometrics, Elsevier, vol. 217(1), pages 46-78.
    3. Paulo, Rui & García-Donato, Gonzalo & Palomo, Jesús, 2012. "Calibration of computer models with multivariate output," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 3959-3974.
    4. Felix Abramovich & Vadim Grinshtein, 2013. "Estimation of a sparse group of sparse vectors," Biometrika, Biometrika Trust, vol. 100(2), pages 355-370.
    5. Hauzenberger, Niko & Huber, Florian & Klieber, Karin, 2023. "Real-time inflation forecasting using non-linear dimension reduction techniques," International Journal of Forecasting, Elsevier, vol. 39(2), pages 901-921.
    6. Hauzenberger Niko & Huber Florian & Koop Gary, 2024. "Dynamic Shrinkage Priors for Large Time-Varying Parameter Regressions Using Scalable Markov Chain Monte Carlo Methods," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 28(2), pages 201-225, April.
    7. Yi Liu & Veronika Ročková & Yuexi Wang, 2021. "Variable selection with ABC Bayesian forests," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(3), pages 453-481, July.
    8. Zhongfang He, 2024. "Locally time-varying parameter regression," Econometric Reviews, Taylor & Francis Journals, vol. 43(5), pages 269-300, May.
    9. Joshua C.C. Chan & Rodney W. Strachan, 2023. "Bayesian State Space Models In Macroeconometrics," Journal of Economic Surveys, Wiley Blackwell, vol. 37(1), pages 58-75, February.
    10. Chen, Sihan & Ming, Lei & Yang, Haoxi & Yang, Shenggang, 2025. "Iterated Dynamic Model Averaging and application to inflation forecasting," International Review of Financial Analysis, Elsevier, vol. 102(C).
    11. Dimitris Korobilis & Kenichi Shimizu, 2022. "Bayesian Approaches to Shrinkage and Sparse Estimation," Foundations and Trends(R) in Econometrics, now publishers, vol. 11(4), pages 230-354, June.
    12. Zheng Fan & Worapree Maneesoonthorn & Yong Song, 2025. "A New Perspective of the Meese-Rogoff Puzzle: Application of Sparse Dynamic Shrinkage," Papers 2507.14408, arXiv.org.
    13. Florian Huber & Gary Koop & Michael Pfarrhofer, 2020. "Bayesian Inference in High-Dimensional Time-varying Parameter Models using Integrated Rotated Gaussian Approximations," Papers 2002.10274, arXiv.org.
    14. Zhou Lan & Brian J. Reich & Joseph Guinness & Dipankar Bandyopadhyay & Liangsuo Ma & F. Gerard Moeller, 2022. "Geostatistical modeling of positive‐definite matrices: An application to diffusion tensor imaging," Biometrics, The International Biometric Society, vol. 78(2), pages 548-559, June.
    15. Martin Feldkircher & Florian Huber & Gary Koop & Michael Pfarrhofer, 2022. "APPROXIMATE BAYESIAN INFERENCE AND FORECASTING IN HUGE‐DIMENSIONAL MULTICOUNTRY VARs," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 63(4), pages 1625-1658, November.
    16. Dimitris Korobilis, 2021. "High-Dimensional Macroeconomic Forecasting Using Message Passing Algorithms," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(2), pages 493-504, March.
    17. Hauzenberger, Niko, 2021. "Flexible Mixture Priors for Large Time-varying Parameter Models," Econometrics and Statistics, Elsevier, vol. 20(C), pages 87-108.
    18. Joshua C. C. Chan, 2018. "Specification tests for time-varying parameter models with stochastic volatility," Econometric Reviews, Taylor & Francis Journals, vol. 37(8), pages 807-823, September.
    19. Bernardi, Mauro & Costola, Michele, 2019. "High-dimensional sparse financial networks through a regularised regression model," SAFE Working Paper Series 244, Leibniz Institute for Financial Research SAFE.
    20. Balcilar, Mehmet & Berisha, Edmond & Gupta, Rangan & Pierdzioch, Christian, 2021. "Time-varying evidence of predictability of financial stress in the United States over a century: The role of inequality," Structural Change and Economic Dynamics, Elsevier, vol. 57(C), pages 87-92.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijds:v:4:y:2025:i:2:p:114-132. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.