Constrained Maximum-Entropy Sampling

A fundamental experimental design problem is to select a most informative subset, having prespecified size, from a set of correlated random variables. Instances of this problem arise in many applied domains such as meteorology, environmental statistics, and statistical geology. In these applications, observations can be collected at different locations and, possibly, at different times. Information is measured by “entropy.” Practical situations have further restrictions on the design space. For example, budgetary limits, geographical considerations, as well as legislative and political considerations may restrict the design space in a complicated manner. Using techniques of linear algebra, combinatorial optimization, and convex optimization, we develop upper and lower bounds on the optimal value for the Gaussian case. We describe how these bounds can be integrated into a branch-and-bound algorithm for the exact solution of these design problems. Finally, we describe how we have implemented this algorithm, and we present computational results for estimated covariance matrices corresponding to sets of environmental monitoring stations in the Ohio Valley of the United States.