IDEAS home Printed from https://ideas.repec.org/a/spr/jglopt/v56y2013i3p1247-1293.html
   My bibliography  Save this article

Derivative-free optimization: a review of algorithms and comparison of software implementations

Author

Listed:
  • Luis Rios
  • Nikolaos Sahinidis

Abstract

This paper addresses the solution of bound-constrained optimization problems using algorithms that require only the availability of objective function values but no derivative information. We refer to these algorithms as derivative-free algorithms. Fueled by a growing number of applications in science and engineering, the development of derivative-free optimization algorithms has long been studied, and it has found renewed interest in recent time. Along with many derivative-free algorithms, many software implementations have also appeared. The paper presents a review of derivative-free algorithms, followed by a systematic comparison of 22 related implementations using a test set of 502 problems. The test bed includes convex and nonconvex problems, smooth as well as nonsmooth problems. The algorithms were tested under the same conditions and ranked under several criteria, including their ability to find near-global solutions for nonconvex problems, improve a given starting point, and refine a near-optimal solution. A total of 112,448 problem instances were solved. We find that the ability of all these solvers to obtain good solutions diminishes with increasing problem size. For the problems used in this study, TOMLAB/MULTIMIN, TOMLAB/GLCCLUSTER, MCS and TOMLAB/LGO are better, on average, than other derivative-free solvers in terms of solution quality within 2,500 function evaluations. These global solvers outperform local solvers even for convex problems. Finally, TOMLAB/OQNLP, NEWUOA, and TOMLAB/MULTIMIN show superior performance in terms of refining a near-optimal solution. Copyright Springer Science+Business Media, LLC. 2013

Suggested Citation

  • Luis Rios & Nikolaos Sahinidis, 2013. "Derivative-free optimization: a review of algorithms and comparison of software implementations," Journal of Global Optimization, Springer, vol. 56(3), pages 1247-1293, July.
    • Handle: RePEc:spr:jglopt:v:56:y:2013:i:3:p:1247-1293
    DOI: 10.1007/s10898-012-9951-y
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10898-012-9951-y
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10898-012-9951-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Robert L. Smith, 1984. "Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions," Operations Research, INFORMS, vol. 32(6), pages 1296-1308, December.
    2. NESTEROV, Yu., 2007. "Gradient methods for minimizing composite objective function," LIDAM Discussion Papers CORE 2007076, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    3. Hvattum, Lars Magnus & Glover, Fred, 2009. "Finding local optima of high-dimensional functions using direct search methods," European Journal of Operational Research, Elsevier, vol. 195(1), pages 31-45, May.
    4. Rommel Regis & Christine Shoemaker, 2005. "Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions," Journal of Global Optimization, Springer, vol. 31(1), pages 153-171, January.
    5. Claude J. P. Bélisle & H. Edwin Romeijn & Robert L. Smith, 1993. "Hit-and-Run Algorithms for Generating Multivariate Distributions," Mathematics of Operations Research, INFORMS, vol. 18(2), pages 255-266, May.
    6. J.E. Orosz & S.H. Jacobson, 2002. "Analysis of Static Simulated Annealing Algorithms," Journal of Optimization Theory and Applications, Springer, vol. 115(1), pages 165-182, October.
    7. Genetha Anne Gray & Tamara G. Kolda & Ken Sale & Malin M. Young, 2004. "Optimizing an Empirical Scoring Function for Transmembrane Protein Structure Determination," INFORMS Journal on Computing, INFORMS, vol. 16(4), pages 406-418, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stephen Baumert & Archis Ghate & Seksan Kiatsupaibul & Yanfang Shen & Robert L. Smith & Zelda B. Zabinsky, 2009. "Discrete Hit-and-Run for Sampling Points from Arbitrary Distributions Over Subsets of Integer Hyperrectangles," Operations Research, INFORMS, vol. 57(3), pages 727-739, June.
    2. Luis V. Montiel & J. Eric Bickel, 2014. "A Generalized Sampling Approach for Multilinear Utility Functions Given Partial Preference Information," Decision Analysis, INFORMS, vol. 11(3), pages 147-170, September.
    3. Sorawit Saengkyongam & Anthony Hayter & Seksan Kiatsupaibul & Wei Liu, 2020. "Efficient computation of the stochastic behavior of partial sum processes," Computational Statistics, Springer, vol. 35(1), pages 343-358, March.
    4. Cyril Bachelard & Apostolos Chalkis & Vissarion Fisikopoulos & Elias Tsigaridas, 2022. "Randomized geometric tools for anomaly detection in stock markets," Papers 2205.03852, arXiv.org, revised May 2022.
    5. Bélisle, Claude, 2000. "Slow hit-and-run sampling," Statistics & Probability Letters, Elsevier, vol. 47(1), pages 33-43, March.
    6. de Klerk, Etienne & Badenbroek, Riley, 2022. "Simulated annealing with hit-and-run for convex optimization: complexity analysis and practical perspectives," Other publications TiSEM 323b4588-65e0-4889-a555-9, Tilburg University, School of Economics and Management.
    7. Nabil Kahalé, 2019. "Efficient Simulation of High Dimensional Gaussian Vectors," Mathematics of Operations Research, INFORMS, vol. 44(1), pages 58-73, February.
    8. Boukouvala, Fani & Misener, Ruth & Floudas, Christodoulos A., 2016. "Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO," European Journal of Operational Research, Elsevier, vol. 252(3), pages 701-727.
    9. Tsionas, Mike G., 2020. "A coherent approach to Bayesian Data Envelopment Analysis," European Journal of Operational Research, Elsevier, vol. 281(2), pages 439-448.
    10. Cyril Bachelard & Apostolos Chalkis & Vissarion Fisikopoulos & Elias Tsigaridas, 2023. "Randomized geometric tools for anomaly detection in stock markets," Post-Print hal-04223511, HAL.
    11. Badenbroek, Riley & de Klerk, Etienne, 2022. "Complexity analysis of a sampling-based interior point method for convex optimization," Other publications TiSEM 3d774c6d-8141-4f31-a621-5, Tilburg University, School of Economics and Management.
    12. Riley Badenbroek & Etienne Klerk, 2022. "Simulated Annealing for Convex Optimization: Rigorous Complexity Analysis and Practical Perspectives," Journal of Optimization Theory and Applications, Springer, vol. 194(2), pages 465-491, August.
    13. Huseyin Mete & Yanfang Shen & Zelda Zabinsky & Seksan Kiatsupaibul & Robert Smith, 2011. "Pattern discrete and mixed Hit-and-Run for global optimization," Journal of Global Optimization, Springer, vol. 50(4), pages 597-627, August.
    14. Luis V. Montiel & J. Eric Bickel, 2013. "Approximating Joint Probability Distributions Given Partial Information," Decision Analysis, INFORMS, vol. 10(1), pages 26-41, March.
    15. Huseyin Onur Mete & Zelda B. Zabinsky, 2014. "Multiobjective Interacting Particle Algorithm for Global Optimization," INFORMS Journal on Computing, INFORMS, vol. 26(3), pages 500-513, August.
    16. Richard J. Caron & Tim Traynor & Shafiu Jibrin, 2010. "Feasibility and Constraint Analysis of Sets of Linear Matrix Inequalities," INFORMS Journal on Computing, INFORMS, vol. 22(1), pages 144-153, February.
    17. Luca Anzilli & Silvio Giove, 2020. "Multi-criteria and medical diagnosis for application to health insurance systems: a general approach through non-additive measures," Decisions in Economics and Finance, Springer;Associazione per la Matematica, vol. 43(2), pages 559-582, December.
    18. Corrente, Salvatore & Figueira, José Rui & Greco, Salvatore, 2014. "The SMAA-PROMETHEE method," European Journal of Operational Research, Elsevier, vol. 239(2), pages 514-522.
    19. Umberto Amato & Anestis Antoniadis & Italia De Feis & Irene Gijbels, 2021. "Penalised robust estimators for sparse and high-dimensional linear models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 1-48, March.
    20. Hoseinzade, Davood & Lakzian, Esmail & Hashemian, Ali, 2021. "A blackbox optimization of volumetric heating rate for reducing the wetness of the steam flow through turbine blades," Energy, Elsevier, vol. 220(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jglopt:v:56:y:2013:i:3:p:1247-1293. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.