IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v30y2018i3p608-624.html
   My bibliography  Save this article

What Works Best When? A Systematic Evaluation of Heuristics for Max-Cut and QUBO

Author

Listed:
  • Iain Dunning

    (DeepMind, London N1C 4AG, United Kingdom)

  • Swati Gupta

    (Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332)

  • John Silberholz

    (Ross School of Business, University of Michigan, Ann Arbor, Michigan 48109)

Abstract

Though empirical testing is broadly used to evaluate heuristics, there are shortcomings with how it is often applied in practice. In a systematic review of Max-Cut and quadratic unconstrained binary optimization (QUBO) heuristics papers, we found only 4% publish source code, only 14% compare heuristics with identical termination criteria, and most experiments are performed with an artificial, homogeneous set of problem instances. To address these limitations, we implement and release as open-source a code-base of 10 Max-Cut and 27 QUBO heuristics. We perform heuristic evaluation using cloud computing on a library of 3,296 instances. This large-scale evaluation provides insight into the types of problem instances for which each heuristic performs well or poorly. Because no single heuristic outperforms all others across all problem instances, we use machine learning to predict which heuristic will work best on a previously unseen problem instance, a key question facing practitioners.

Suggested Citation

  • Iain Dunning & Swati Gupta & John Silberholz, 2018. "What Works Best When? A Systematic Evaluation of Heuristics for Max-Cut and QUBO," INFORMS Journal on Computing, INFORMS, vol. 30(3), pages 608-624, August.
    • Handle: RePEc:inm:orijoc:v:30:y:2018:i:3:p:608-624
    DOI: 10.1287/ijoc.2017.0798
    as

    Download full text from publisher

    File URL: https://doi.org/10.1287/ijoc.2017.0798
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2017.0798?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Raymond R. Hill & Charles H. Reilly, 2000. "The Effects of Coefficient Correlation Structure in Two-Dimensional Knapsack Problems on Solution Procedure Performance," Management Science, INFORMS, vol. 46(2), pages 302-317, February.
    2. Edmund K. Burke & Matthew Hyde & Graham Kendall & Gabriela Ochoa & Ender Özcan & John R. Woodward, 2010. "A Classification of Hyper-heuristic Approaches," International Series in Operations Research & Management Science, in: Michel Gendreau & Jean-Yves Potvin (ed.), Handbook of Metaheuristics, chapter 0, pages 449-468, Springer.
    3. John Silberholz & Bruce Golden, 2010. "Comparison of Metaheuristics," International Series in Operations Research & Management Science, in: Michel Gendreau & Jean-Yves Potvin (ed.), Handbook of Metaheuristics, chapter 0, pages 625-640, Springer.
    4. Thomas E. Vollmann & Elwood S. Buffa, 1966. "The Facilities Layout Problem in Perspective," Management Science, INFORMS, vol. 12(10), pages 450-468, June.
    5. Rafael Martí & Abraham Duarte & Manuel Laguna, 2009. "Advanced Scatter Search for the Max-Cut Problem," INFORMS Journal on Computing, INFORMS, vol. 21(1), pages 26-38, February.
    6. Francisco Barahona & Martin Grötschel & Michael Jünger & Gerhard Reinelt, 1988. "An Application of Combinatorial Optimization to Statistical Physics and Circuit Layout Design," Operations Research, INFORMS, vol. 36(3), pages 493-513, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fred Glover & Gary Kochenberger & Rick Hennig & Yu Du, 2022. "Quantum bridge analytics I: a tutorial on formulating and using QUBO models," Annals of Operations Research, Springer, vol. 314(1), pages 141-183, July.
    2. Ricardo N. Liang & Eduardo A. J. Anacleto & Cláudio N. Meneses, 2022. "Data structures for speeding up Tabu Search when solving sparse quadratic unconstrained binary optimization problems," Journal of Heuristics, Springer, vol. 28(4), pages 433-479, August.
    3. Cheng Lu & Zhibin Deng & Shu-Cherng Fang & Wenxun Xing, 2023. "A New Global Algorithm for Max-Cut Problem with Chordal Sparsity," Journal of Optimization Theory and Applications, Springer, vol. 197(2), pages 608-638, May.
    4. Timotej Hrga & Janez Povh, 2021. "MADAM: a parallel exact solver for max-cut based on semidefinite programming and ADMM," Computational Optimization and Applications, Springer, vol. 80(2), pages 347-375, November.
    5. Theja Tulabandhula & Deeksha Sinha & Saketh Reddy Karra & Prasoon Patidar, 2020. "Multi-Purchase Behavior: Modeling, Estimation and Optimization," Papers 2006.08055, arXiv.org, revised Aug 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fuda Ma & Jin-Kao Hao, 2017. "A multiple search operator heuristic for the max-k-cut problem," Annals of Operations Research, Springer, vol. 248(1), pages 365-403, January.
    2. Wenxing Zhu & Geng Lin & M. M. Ali, 2013. "Max- k -Cut by the Discrete Dynamic Convexized Method," INFORMS Journal on Computing, INFORMS, vol. 25(1), pages 27-40, February.
    3. Marko Ɖurasević & Domagoj Jakobović, 2019. "Creating dispatching rules by simple ensemble combination," Journal of Heuristics, Springer, vol. 25(6), pages 959-1013, December.
    4. Gahm, Christian & Uzunoglu, Aykut & Wahl, Stefan & Ganschinietz, Chantal & Tuma, Axel, 2022. "Applying machine learning for the anticipation of complex nesting solutions in hierarchical production planning," European Journal of Operational Research, Elsevier, vol. 296(3), pages 819-836.
    5. Dell'Amico, Mauro & Trubian, Marco, 1998. "Solution of large weighted equicut problems," European Journal of Operational Research, Elsevier, vol. 106(2-3), pages 500-521, April.
    6. Carlos Contreras Bolton & Gustavo Gatica & Víctor Parada, 2013. "Automatically Generated Algorithms for the Vertex Coloring Problem," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-9, March.
    7. Goldengorin, Boris, 2009. "Maximization of submodular functions: Theory and enumeration algorithms," European Journal of Operational Research, Elsevier, vol. 198(1), pages 102-112, October.
    8. Bolte, Andreas & Thonemann, Ulrich Wilhelm, 1996. "Optimizing simulated annealing schedules with genetic programming," European Journal of Operational Research, Elsevier, vol. 92(2), pages 402-416, July.
    9. Xunzhao Yin & Yu Qian & Alptekin Vardar & Marcel Günther & Franz Müller & Nellie Laleni & Zijian Zhao & Zhouhang Jiang & Zhiguo Shi & Yiyu Shi & Xiao Gong & Cheng Zhuo & Thomas Kämpfe & Kai Ni, 2024. "Ferroelectric compute-in-memory annealer for combinatorial optimization problems," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    10. Cheng Lu & Zhibin Deng, 2021. "A branch-and-bound algorithm for solving max-k-cut problem," Journal of Global Optimization, Springer, vol. 81(2), pages 367-389, October.
    11. Jooken, Jorik & Leyman, Pieter & De Causmaecker, Patrick, 2023. "Features for the 0-1 knapsack problem based on inclusionwise maximal solutions," European Journal of Operational Research, Elsevier, vol. 311(1), pages 36-55.
    12. Kallestad, Jakob & Hasibi, Ramin & Hemmati, Ahmad & Sörensen, Kenneth, 2023. "A general deep reinforcement learning hyperheuristic framework for solving combinatorial optimization problems," European Journal of Operational Research, Elsevier, vol. 309(1), pages 446-468.
    13. Gary Kochenberger & Jin-Kao Hao & Fred Glover & Mark Lewis & Zhipeng Lü & Haibo Wang & Yang Wang, 2014. "The unconstrained binary quadratic programming problem: a survey," Journal of Combinatorial Optimization, Springer, vol. 28(1), pages 58-81, July.
    14. Jorge A. Sefair & Oscar Guaje & Andrés L. Medaglia, 2021. "A column-oriented optimization approach for the generation of correlated random vectors," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 43(3), pages 777-808, September.
    15. Shenshen Gu & Yue Yang, 2020. "A Deep Learning Algorithm for the Max-Cut Problem Based on Pointer Network Structure with Supervised Learning and Reinforcement Learning Strategies," Mathematics, MDPI, vol. 8(2), pages 1-20, February.
    16. Wang, Yang & Lü, Zhipeng & Glover, Fred & Hao, Jin-Kao, 2012. "Path relinking for unconstrained binary quadratic programming," European Journal of Operational Research, Elsevier, vol. 223(3), pages 595-604.
    17. Drexl, Michael & Schneider, Michael, 2015. "A survey of variants and extensions of the location-routing problem," European Journal of Operational Research, Elsevier, vol. 241(2), pages 283-308.
    18. Aleksandra Swiercz & Wojciech Frohmberg & Michal Kierzynka & Pawel Wojciechowski & Piotr Zurkowski & Jan Badura & Artur Laskowski & Marta Kasprzak & Jacek Blazewicz, 2018. "GRASShopPER—An algorithm for de novo assembly based on GPU alignments," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-23, August.
    19. repec:dgr:rugsom:99a17 is not listed on IDEAS
    20. Lu, Yongliang & Benlic, Una & Wu, Qinghua, 2018. "A memetic algorithm for the Orienteering Problem with Mandatory Visits and Exclusionary Constraints," European Journal of Operational Research, Elsevier, vol. 268(1), pages 54-69.
    21. Leo Lopes & Kate Smith-Miles, 2013. "Generating Applicable Synthetic Instances for Branch Problems," Operations Research, INFORMS, vol. 61(3), pages 563-577, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:30:y:2018:i:3:p:608-624. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.