On the performance of joint linear minimum mean squared error (LMMSE) filtering and parameter estimation

We consider the problem of LMMSE estimation (such as Wiener and Kalman filtering) in the presence of a number of unknown parameters in the second-order statistics, that need to be estimated also. This well-known joint filtering and parameter estimation problem has numerous applications. It is a hybrid estimation problem in which the signal to be estimated by linear filtering is random, and the unknown parameters are deterministic. As the signal is random, it can also be eliminated, allowing parameter estimation from the marginal distribution of the data. An intriguing question is then the relative performance of joint vs. marginalized parameter estimation. In this paper, we consider jointly Gaussian signal and data and we first provide contributions to Cramer-Rao bounds (CRBs). We characterize the difference between the Hybrid Fisher Information Matrix (HFIM) and the classical marginalized FIM on the one hand, and between the FIM (with CRB asymptotically attained by ML) and the popular Modified FIM (MFIM, inverse of Modified CRB) which is a loose bound. We then investigate three iterative (alternating optimization) joint estimation approaches: Alternating Maximum A Posteriori for Signal and Maximum Likelihood for parameters (AMAPML), which in spite of a better HFIM suffers from inconsistent parameter bias, Expectation-Maximization (EM) which converges to (marginalized) ML (but with AMAPML signal estimate), and Variational Bayes (VB) which yields an improved signal estimate with the parameter estimate asymptotically becoming ML.