Time‐inhomogeneous discrete stochastic search methods for optimal Bernoulli parameters

We present two time‐inhomogeneous search processes for finding the optimal Bernoulli parameters, where the performance measure cannot be evaluated exactly but must be estimated through Monte Carlo simulation. At each iteration, two neighbouring alternatives are compared and the one that appears to be better is passed on to the next iteration. The first search process uses an increasing sample size of each configuration at each iteration. The second search process uses a sequential sampling procedure with increasing boundaries as the number of iterations increases. At each iteration the acceptance of a new configuration depends on the iterate number, therefore, the search process turns out to be inhomogeneous Markov chain. We show that if the increase occurs slower than a certain rate, these search processes will converge to the optimal set with probability one. © 1998 John Wiley & Sons, Ltd.