A universal strong law of large numbers for conditional expectations via nearest neighbors

For k n -nearest neighbor estimates of a regression Y on X (d-dimensional random vector X, integrable real random variable Y) based on observed independent copies of ( X , Y ) , strong universal pointwise consistency is shown, i.e., strong consistency P X -almost everywhere for general distribution of ( X , Y ) . With tie-breaking by indices, this means validity of a universal strong law of large numbers for conditional expectations E ( Y | X = x ) .