Robustness via a Tradeoff Between Fisher Information and Relative Entropy

We look at the problem of finding the worst-case distribution in a convex family of distributions defined as those whose relative entropy, relative to a nominal distribution, is less than a threshold. The worst-case distribution is selected as the one whose Fisher information for the mean of the distribution is the lowest. This problem is connected to a penalized maximum likelihood estimation problem. We present a novel algorithm for computing this worst-case (robust) distribution, show implementation results and analyze properties of the robust distribution.