Optimal Sequential Design in a Controlled Non‐parametric Regression

. In a non‐parametric regression, the heteroscedasticity (dependence of the variance of the regression error on the predictor) can be a serious complication in estimation or visualization of an underlying regression function. If a controlled sampling is permitted, then the statistician can choose the design of predictors which attenuates the effect of heteroscedasticity. It is proposed to use a design which minimizes the mean integrated squared error of the regression function estimation. Then the corresponding optimal design density is proportional to the standard deviation of the regression error (the so‐called scale function). Because in general the statistician does not know an underlying scale function, the natural question is as follows: is it possible to suggest a sequential design which performs as well as an oracle that knows the underlying scale function? The answer is ‘yes’, and a corresponding sequential procedure is developed. It is proved, for the first time in the literature, that a data‐driven sequential design, together with an adaptive regression estimator, can mimic the oracle and be sharp minimax. Further, it is shown that the suggested method is feasible for small samples.