Resampling-based calculation of the information matrix for general identification problems

The asymptotic normality of maximum likelihood and other general estimation schemes provide a powerful method for determining statistical uncertainty bounds for the resulting estimates. This asymptotic normality result depends critically on the inverse Fisher information matrix as an approximation to the covariance matrix. Unfortunately, the Fisher information matrix is difficult to obtain in a large fraction of practical problems. The paper presents a relatively simple method for computing the Fisher information matrix based on a combination of Hessian matrix estimation and a computer-based resampling technique for averaging the Hessians. The Hessian estimation can be performed using either loss function values alone or, if available, values for the gradient of the loss function. The approach is demonstrated on a mid-sized estimation problem