IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v35y2023i2p509-517.html
   My bibliography  Save this article

Scalable Parallel Nonlinear Optimization with PyNumero and Parapint

Author

Listed:
  • Jose S. Rodriguez

    (Davidson School of Chemical Engineering, Purdue University, West Lafayette, Indiana 47907)

  • Robert B. Parker

    (Center for Computing Research, Sandia National Laboratories, Albuquerque, New Mexico 87123; Carnegie Mellon University, Department of Chemical Engineering, Pittsburgh, Pennsylvania 15213)

  • Carl D. Laird

    (Carnegie Mellon University, Department of Chemical Engineering, Pittsburgh, Pennsylvania 15213)

  • Bethany L. Nicholson

    (Center for Computing Research, Sandia National Laboratories, Albuquerque, New Mexico 87123)

  • John D. Siirola

    (Center for Computing Research, Sandia National Laboratories, Albuquerque, New Mexico 87123)

  • Michael L. Bynum

    (Center for Computing Research, Sandia National Laboratories, Albuquerque, New Mexico 87123)

Abstract

We describe PyNumero, an open-source, object-oriented programming framework in Python that supports rapid development of performant parallel algorithms for structured nonlinear programming problems (NLP’s) using the Message Passing Interface (MPI). PyNumero provides three fundamental building blocks for developing NLP algorithms: a fast interface for calculating first and second derivatives with the AMPL Solver Library (ASL), a number of interfaces to efficient linear solvers, and block-structured vectors and matrices based on NumPy, SciPy, and MPI that support distributed parallel storage and computation. PyNumero’s design enables efficient, parallel algorithm development using high-level Python syntax while keeping expensive numerical calculations in fast, compiled implementations based on languages like C and Fortran. To demonstrate the utility of PyNumero, we also present Parapint, a Python package built on PyNumero for parallel solution of dynamic optimization problems. Parapint includes a parallel interior-point solver based on Schur-Complement decomposition. We illustrate the effectiveness of PyNumero for developing parallel algorithms with both code examples and scalability analyses for parallel matrix-vector dot products, parallel solution of structured systems of linear equations using Schur-Complement decomposition, and the parallel solution of a two-dimensional PDE optimal control problem. Our numerical results show nearly perfect scaling to more than 1,000 cores for large matrix-vector dot products and structured linear systems. Moreover, we obtain more than 354 times speedup for the optimal control example.

Suggested Citation

  • Jose S. Rodriguez & Robert B. Parker & Carl D. Laird & Bethany L. Nicholson & John D. Siirola & Michael L. Bynum, 2023. "Scalable Parallel Nonlinear Optimization with PyNumero and Parapint," INFORMS Journal on Computing, INFORMS, vol. 35(2), pages 509-517, March.
    • Handle: RePEc:inm:orijoc:v:35:y:2023:i:2:p:509-517
    DOI: 10.1287/ijoc.2023.1272
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/ijoc.2023.1272
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2023.1272?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Daniel Word & Jia Kang & Johan Akesson & Carl Laird, 2014. "Efficient parallel solution of large-scale nonlinear dynamic optimization problems," Computational Optimization and Applications, Springer, vol. 59(3), pages 667-688, December.
    2. Charles R. Harris & K. Jarrod Millman & Stéfan J. Walt & Ralf Gommers & Pauli Virtanen & David Cournapeau & Eric Wieser & Julian Taylor & Sebastian Berg & Nathaniel J. Smith & Robert Kern & Matti Picu, 2020. "Array programming with NumPy," Nature, Nature, vol. 585(7825), pages 357-362, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tan Wang & L. Jeff Hong, 2023. "Large-Scale Inventory Optimization: A Recurrent Neural Networks–Inspired Simulation Approach," INFORMS Journal on Computing, INFORMS, vol. 35(1), pages 196-215, January.
    2. Léon Faure & Bastien Mollet & Wolfram Liebermeister & Jean-Loup Faulon, 2023. "A neural-mechanistic hybrid approach improving the predictive power of genome-scale metabolic models," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    3. Claudia Quinteros-Cartaya & Guillermo Solorio-Magaña & Francisco Javier Núñez-Cornú & Felipe de Jesús Escalona-Alcázar & Diana Núñez, 2023. "Microearthquakes in the Guadalajara Metropolitan Zone, Mexico: evidence from buried active faults in Tesistán Valley, Zapopan," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 116(3), pages 2797-2818, April.
    4. López Pérez, Mario & Mansilla Corona, Ricardo, 2022. "Ordinal synchronization and typical states in high-frequency digital markets," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 598(C).
    5. Jessica M. Vanslambrouck & Sean B. Wilson & Ker Sin Tan & Ella Groenewegen & Rajeev Rudraraju & Jessica Neil & Kynan T. Lawlor & Sophia Mah & Michelle Scurr & Sara E. Howden & Kanta Subbarao & Melissa, 2022. "Enhanced metanephric specification to functional proximal tubule enables toxicity screening and infectious disease modelling in kidney organoids," Nature Communications, Nature, vol. 13(1), pages 1-23, December.
    6. Kiran Krishnamachari & Dylan Lu & Alexander Swift-Scott & Anuar Yeraliyev & Kayla Lee & Weitai Huang & Sim Ngak Leng & Anders Jacobsen Skanderup, 2022. "Accurate somatic variant detection using weakly supervised deep learning," Nature Communications, Nature, vol. 13(1), pages 1-8, December.
    7. Lauren L. Porter & Allen K. Kim & Swechha Rimal & Loren L. Looger & Ananya Majumdar & Brett D. Mensh & Mary R. Starich & Marie-Paule Strub, 2022. "Many dissimilar NusG protein domains switch between α-helix and β-sheet folds," Nature Communications, Nature, vol. 13(1), pages 1-12, December.
    8. Matthew Rosenblatt & Link Tejavibulya & Rongtao Jiang & Stephanie Noble & Dustin Scheinost, 2024. "Data leakage inflates prediction performance in connectome-based machine learning models," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    9. Jackie Grant & Mark Hindmarsh & Sergey E. Koposov, 2022. "The distribution of loss to future USS pensions due to the UUK cuts of April 2022," Papers 2206.06201, arXiv.org.
    10. Sayedali Shetab Boushehri & Katharina Essig & Nikolaos-Kosmas Chlis & Sylvia Herter & Marina Bacac & Fabian J. Theis & Elke Glasmacher & Carsten Marr & Fabian Schmich, 2023. "Explainable machine learning for profiling the immunological synapse and functional characterization of therapeutic antibodies," Nature Communications, Nature, vol. 14(1), pages 1-16, December.
    11. Bethany L. Nicholson & Wei Wan & Shivakumar Kameswaran & Lorenz T. Biegler, 2018. "Parallel cyclic reduction strategies for linear systems that arise in dynamic optimization problems," Computational Optimization and Applications, Springer, vol. 70(2), pages 321-350, June.
    12. Shukla, Mohak & Thakur, Ajay D., 2022. "An Enquiry on similarities between Renormalization Group and Auto-Encoders using Transfer Learning," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 608(P1).
    13. Khaled Akkad & David He, 2023. "A dynamic mode decomposition based deep learning technique for prognostics," Journal of Intelligent Manufacturing, Springer, vol. 34(5), pages 2207-2224, June.
    14. Romain Fournier & Zoi Tsangalidou & David Reich & Pier Francesco Palamara, 2023. "Haplotype-based inference of recent effective population size in modern and ancient DNA samples," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    15. Laura Portell & Sergi Morera & Helena Ramalhinho, 2022. "Door-to-Door Transportation Services for Reduced Mobility Population: A Descriptive Analytics of the City of Barcelona," IJERPH, MDPI, vol. 19(8), pages 1-20, April.
    16. Caroline Haimerl & Douglas A. Ruff & Marlene R. Cohen & Cristina Savin & Eero P. Simoncelli, 2023. "Targeted V1 comodulation supports task-adaptive sensory decisions," Nature Communications, Nature, vol. 14(1), pages 1-15, December.
    17. Jonas Bunsen & Matthias Finkbeiner, 2022. "An Introductory Review of Input-Output Analysis in Sustainability Sciences Including Potential Implications of Aggregation," Sustainability, MDPI, vol. 15(1), pages 1-24, December.
    18. Petros C. Lazaridis & Ioannis E. Kavvadias & Konstantinos Demertzis & Lazaros Iliadis & Lazaros K. Vasiliadis, 2023. "Interpretable Machine Learning for Assessing the Cumulative Damage of a Reinforced Concrete Frame Induced by Seismic Sequences," Sustainability, MDPI, vol. 15(17), pages 1-31, August.
    19. Matthias Wagener & Andriette Bekker & Mohammad Arashi, 2021. "Mastering the Body and Tail Shape of a Distribution," Mathematics, MDPI, vol. 9(21), pages 1-22, October.
    20. Gallo Cassarino, Tiziano & Barrett, Mark, 2022. "Meeting UK heat demands in zero emission renewable energy systems using storage and interconnectors," Applied Energy, Elsevier, vol. 306(PB).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:35:y:2023:i:2:p:509-517. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.