Steady State and Transient MSE Analysis of Convexly Constrained Mixture Methods

We investigate convexly constrained mixture methods to adaptively combine outputs of two adaptive filters running in parallel to model a desired unknown system. We compare several algorithms with respect to their mean-square error in the steady state, when the underlying unknown system is nonstationary with a random walk model. We demonstrate that these algorithms are universal such that they achieve the performance of the best constituent filter in the steady state if certain algorithmic parameters are chosen properly. We also demonstrate that certain mixtures converge to the optimal convex combination filter such that their steady-state performances can be better than the best constituent filter. We also perform the transient analysis of these updates in the mean and mean-square error sense. Furthermore, we show that the investigated convexly constrained algorithms update certain auxiliary variables through sigmoid nonlinearity, hence, in this sense, related.