Author
Listed:
- C. Coelho
- M. Fernanda P. Costa
- L.L. Ferrás
Abstract
Real‐world systems are often formulated as constrained optimization problems. Techniques to incorporate constraints into neural networks (NN), such as neural ordinary differential equations (Neural ODEs), have been used. However, these introduce hyperparameters that require manual tuning through trial and error, raising doubts about the successful incorporation of constraints into the generated model. This paper describes in detail the two‐stage training method for Neural ODEs, a simple, effective, and penalty parameter‐free approach to model constrained systems. In this approach, the constrained optimization problem is rewritten as two optimization subproblems that are solved in two stages. The first stage aims at finding feasible NN parameters by minimizing a measure of constraints violation. The second stage aims to find the optimal NN parameters by minimizing the loss function while keeping inside the feasible region. We experimentally demonstrate that our method produces models that satisfy the constraints and also improves their predictive performance, thus ensuring compliance with critical system properties and also contributing to reducing data quantity requirements. Furthermore, we show that the proposed method improves the convergence to an optimal solution and improves the explainability of Neural ODE models. Our proposed two‐stage training method can be used with any NN architectures.
Suggested Citation
C. Coelho & M. Fernanda P. Costa & L.L. Ferrás, 2025.
"A Two‐Stage Training Method for Modeling Constrained Systems With Neural Networks,"
Journal of Forecasting, John Wiley & Sons, Ltd., vol. 44(5), pages 1785-1805, August.
Handle:
RePEc:wly:jforec:v:44:y:2025:i:5:p:1785-1805
DOI: 10.1002/for.3270
Download full text from publisher
Corrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:jforec:v:44:y:2025:i:5:p:1785-1805. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
We have no bibliographic references for this item. You can help adding them by using this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .
Please note that corrections may take a couple of weeks to filter through
the various RePEc services.