A similarity-based approach to prediction

Assume we are asked to predict a real-valued variable y t based on certain characteristics x t = ( x t 1 , … , x t d ) , and on a database consisting of ( x i 1 , … , x i d , y i ) for i = 1 , … , n . Analogical reasoning suggests to combine past observations of x and y with the current values of x to generate an assessment of y by similarity-weighted averaging. Specifically, the predicted value of y , y t s , is the weighted average of all previously observed values y i , where the weight of y i , for every i = 1 , … , n , is the similarity between the vector x t 1 , … , x t d , associated with y t , and the previously observed vector, x i 1 , … , x i d . The “empirical similarity” approach suggests estimation of the similarity function from past data. We discuss this approach as a statistical method of prediction, study its relationship to the statistical literature, and extend it to the estimation of probabilities and of density functions.