A similarity-based approach to prediction

Assume we are asked to predict a real-valued variable yt based on certain characteristics View the MathML source, and on a database consisting of View the MathML source for i=1,...,n. Analogical reasoning suggests to combine past observations of x and y with the current values of x to generate an assessment of y by similarity-weighted averaging. Specifically, the predicted value of y, View the MathML source, is the weighted average of all previously observed values yi, where the weight of yi, for every i=1,...,n, is the similarity between the vector View the MathML source, associated with yt, and the previously observed vector, View the MathML source. The "empirical similarity" approach suggests estimation of the similarity function from past data. We discuss this approach as a statistical method of prediction, study its relationship to the statistical literature, and extend it to the estimation of probabilities and of density functions.