IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v39y2024i6d10.1007_s00180-023-01424-7.html
   My bibliography  Save this article

Combination of optimization-free kriging models for high-dimensional problems

Author

Listed:
  • Tanguy Appriou

    (Univ Clermont Auvergne
    Stellantis, Centre Technique Velizy)

  • Didier Rullière

    (Univ Clermont Auvergne)

  • David Gaudrie

    (Stellantis, Centre Technique Velizy)

Abstract

Kriging metamodeling (also called Gaussian Process regression) is a popular approach to predict the output of a function based on few observations. The Kriging method involves length-scale hyperparameters whose optimization is essential to obtain an accurate model and is typically performed using maximum likelihood estimation (MLE). However, for high-dimensional problems, the hyperparameter optimization is problematic and often fails to provide correct values. This is especially true for Kriging-based design optimization where the dimension is often quite high. In this article, we propose a method for building high-dimensional surrogate models which avoids the hyperparameter optimization by combining Kriging sub-models with randomly chosen length-scales. Contrarily to other approaches, it does not rely on dimension reduction techniques and it provides a closed-form expression for the model. We present a recipe to determine a suitable range for the sub-models length-scales. We also compare different approaches to compute the weights in the combination. We show for a high-dimensional test problem and a real-world application that our combination is more accurate than the classical Kriging approach using MLE.

Suggested Citation

  • Tanguy Appriou & Didier Rullière & David Gaudrie, 2024. "Combination of optimization-free kriging models for high-dimensional problems," Computational Statistics, Springer, vol. 39(6), pages 3049-3071, September.
  • Handle: RePEc:spr:compst:v:39:y:2024:i:6:d:10.1007_s00180-023-01424-7
    DOI: 10.1007/s00180-023-01424-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-023-01424-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00180-023-01424-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bachoc, François, 2013. "Cross Validation and Maximum Likelihood estimations of hyper-parameters of Gaussian processes with model misspecification," Computational Statistics & Data Analysis, Elsevier, vol. 66(C), pages 55-69.
    2. Roustant, Olivier & Ginsbourger, David & Deville, Yves, 2012. "DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 51(i01).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Betancourt, José & Bachoc, François & Klein, Thierry & Idier, Déborah & Pedreros, Rodrigo & Rohmer, Jérémy, 2020. "Gaussian process metamodeling of functional-input code for coastal flood hazard assessment," Reliability Engineering and System Safety, Elsevier, vol. 198(C).
    2. Cousin, Areski & Maatouk, Hassan & Rullière, Didier, 2016. "Kriging of financial term-structures," European Journal of Operational Research, Elsevier, vol. 255(2), pages 631-648.
    3. Binois, M. & Ginsbourger, D. & Roustant, O., 2015. "Quantifying uncertainty on Pareto fronts with Gaussian process conditional simulations," European Journal of Operational Research, Elsevier, vol. 243(2), pages 386-394.
    4. Bachoc, François & Bevilacqua, Moreno & Velandia, Daira, 2019. "Composite likelihood estimation for a Gaussian process under fixed domain asymptotics," Journal of Multivariate Analysis, Elsevier, vol. 174(C).
    5. Cousin, Areski & Maatouk, Hassan & Rullière, Didier, 2016. "Kriging of financial term-structures," European Journal of Operational Research, Elsevier, vol. 255(2), pages 631-648.
    6. Ehsan Mehdad & Jack P. C. Kleijnen, 2018. "Efficient global optimisation for black-box simulation via sequential intrinsic Kriging," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 69(11), pages 1725-1737, November.
    7. Kleijnen, Jack P.C. & Mehdad, E., 2014. "Multivariate Versus Univariate Kriging Metamodels for Multi-Response Simulation Models (Revision of 2012-039)," Discussion Paper 2014-012, Tilburg University, Center for Economic Research.
    8. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    9. G. Dosi & M. C. Pereira & M. E. Virgillito, 2018. "On the robustness of the fat-tailed distribution of firm growth rates: a global sensitivity analysis," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 173-193, April.
    10. Arvind Krishna & Huan Tran & Chaofan Huang & Rampi Ramprasad & V. Roshan Joseph, 2024. "Adaptive Exploration and Optimization of Materials Crystal Structures," INFORMS Joural on Data Science, INFORMS, vol. 3(1), pages 68-83, April.
    11. Diariétou Sambakhé & Lauriane Rouan & Jean-Noël Bacro & Eric Gozé, 2019. "Conditional optimization of a noisy function using a kriging metamodel," Journal of Global Optimization, Springer, vol. 73(3), pages 615-636, March.
    12. Michael Ludkovski & James Risk, 2017. "Sequential Design and Spatial Modeling for Portfolio Tail Risk Measurement," Papers 1710.05204, arXiv.org, revised May 2018.
    13. Gerber, Florian & Nychka, Douglas W., 2021. "Parallel cross-validation: A scalable fitting method for Gaussian process models," Computational Statistics & Data Analysis, Elsevier, vol. 155(C).
    14. Kleijnen, J.P.C. & Mehdad, Ehsan, 2015. "Estimating the Variance of the Predictor in Stochastic Kriging," Other publications TiSEM dbbd2fa2-eccf-4f71-be9b-c, Tilburg University, School of Economics and Management.
    15. Biewen, Martin & Kugler, Philipp, 2021. "Two-stage least squares random forests with an application to Angrist and Evans (1998)," Economics Letters, Elsevier, vol. 204(C).
    16. Giovanni Dosi & Marcelo C. Pereira & Andrea Roventini & Maria Enrica Virgillito, 2022. "A complexity view on the future of work. Meta-modelling exploration of the multi-sector K+S agent based model," LEM Papers Series 2022/22, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    17. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    18. Olgun Aydin & Bartłomiej Igliński & Krzysztof Krukowski & Marek Siemiński, 2022. "Analyzing Wind Energy Potential Using Efficient Global Optimization: A Case Study for the City Gdańsk in Poland," Energies, MDPI, vol. 15(9), pages 1-22, April.
    19. Mickaël Binois & David Ginsbourger & Olivier Roustant, 2020. "On the choice of the low-dimensional domain for global optimization via random embeddings," Journal of Global Optimization, Springer, vol. 76(1), pages 69-90, January.
    20. Mostafa Reisi Gahrooei & Hao Yan & Kamran Paynabar, 2020. "Comments on: On Active Learning Methods for Manifold Data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 29(1), pages 38-41, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:39:y:2024:i:6:d:10.1007_s00180-023-01424-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.