Stopping small-sample stochastic approximation

The practical application of stochastic approximation methods requires a reliable means to stop the iterative process when the estimate is close to the optimizer or when further improvement in the estimate is doubtful. Conventional ideas on stopping stochastic approximation algorithms employ criteria based on a proxy distribution - usually the asymptotic distribution. Yet difficulties may arise when applying such distributions to small (finite) samples. We propose an approach that uses the distribution of a statistically similar process called a surrogate for the proxy distribution rather than the asymptotic distribution. Under certain conditions, surrogate-based probability calculations are close to the actual probabilities. The question of how surrogate processes may be developed is also addressed. Two example applications are given.