Gaussian Assumption: The Least Favorable but the Most Useful [Lecture Notes]

Gaussian assumption is the most well-known and widely used distribution in many fields such as engineering, statistics, and physics. One of the major reasons why the Gaussian distribution has become so prominent is because of the central limit theorem (CLT) and the fact that the distribution of noise in numerous engineering systems is well captured by the Gaussian distribution. Moreover, features such as analytical tractability and easy generation of other distributions from the Gaussian distribution contributed further to the popularity of Gaussian distribution. Especially, when there is no information about the distribution of observations, Gaussian assumption appears as the most conservative choice. This follows from the fact that the Gaussian distribution minimizes the Fisher information, which is the inverse of the Cramer-Rao lower bound (CRLB) (or equivalently stated, the Gaussian distribution maximizes the CRLB). Therefore, any optimization based on the CRLB under the Gaussian assumption can be considered to be min-max optimal in the sense of minimizing the largest CRLB (see [1] and the references cited therein).