Bivariate Decision Processes

Summary Models are developed for decision making where a system’s evolution is described by a general stochastic process. The general structure of the problem includes many statistical tests such as treatment comparisons, regression models and likelihood ratio tests. The process is monitored and decisions are made in response to the observed system state. The decision process is simplified by using an associated process as well as the underlying state as decision variables; in many situations a functional of the underlying process defines a statistic. The approach is motivated by the idea of a performance metric based on the system state. The bivariate approach allows a wide class of models to be considered and can incorporate long term memory within a simple probability structure. The decisions in this study are based on an average cost and a life-cycle cost. The approach can deal with decisions that entail restarting the process as new or continuing the process after an intervention which changes the system state. The resulting optimization problem solved either by a renewal-reward argument or by a dynamic programming formulation.