A Simple Formula for Mixing Estimators With Different Convergence Rates

Suppose that two estimators, and , are available for estimating an unknown parameter θ, and are known to have convergence rates n -super-1/2 and r n = o ( n -super-1/2), respectively, based on a sample of size n . Typically, the more efficient estimator is less robust than , and a definitive choice cannot be easily made between them under practical circumstances. We propose a simple mixture estimator, in the form of a linear combination of and , which successfully reaps the benefits of both estimators. We prove that the mixture estimator possesses a kind of oracle property so that it captures the fast n -super-1/2 convergence rate of when conditions are favorable, and is at least r n -consistent otherwise. Applications of the mixture estimator are illustrated with examples drawn from different problem settings including orthogonal function regression, local polynomial regression, density derivative estimation, and bootstrap inferences for possibly dependent data.