Integrated estimate-and-optimize decision trees learning for two-stage linear decision-making problems

Several decision-making under uncertainty problems found in industry and the scientific community can be framed as stochastic programs. Traditionally, these problems are addressed using a sequential two-step process, referred to as predict/estimate-then-optimize, in which a predictive distribution of the uncertain parameters is firstly estimated and then used to prescribe a decision. However, most predictive methods focus on minimizing forecast error, without accounting for its impact on decision quality. Moreover, practitioners often emphasize that their main goal is to obtain near-optimal solutions with minimum decision error, rather than least-error predictions. Therefore, in this work, we discuss a new framework for integrating prediction and prescription into the predictive distribution estimation process to be subsequently used to devise a decision. We particularly focus on decision trees and study decision-making problems representable as contextual two-stage linear programs. Firstly, we propose a workable framework along with a non-convex optimization model to account for the impact of the underlying decision-making problem on the predictive distribution estimation process. Then, we recast the non-convex model as a Mixed-Integer Programming (MIP) problem. Acknowledging the difficulty of the MIP reformulation to scale to large-scale instances, we devise a computationally efficient Heuristic strategy for the estimation problem leveraging the structure intrinsic to decision trees. A key feature of the proposed decision-making framework is its ability to instantly assess decisions by mapping new contexts to a leaf and retrieving the precomputed solution of the corresponding two-stage problem. A set of numerical experiments is conducted to illustrate the capability and effectiveness of the proposed framework using three distinct two-stage decision-making problems. We benchmark the proposed approach against prescriptions devised by various alternative frameworks. Five predict/estimate-then-optimize benchmarks that rely on commonly used predictive and distribution estimation methods and three benchmarks based on integrated predict-and-optimize decision-making processes are considered. We focus on evaluating solution quality and the computational performance of the MIP reformulation.