Title :
The α-EM algorithm: surrogate likelihood maximization using α-logarithmic information measures
Author :
Matsuyama, Yasuo
Author_Institution :
Dept. of Comput. Sci., Waseda Univ., Tokyo, Japan
fDate :
3/1/2003 12:00:00 AM
Abstract :
A new likelihood maximization algorithm called the α-EM algorithm (α-expectation-maximization algorithm) is presented. This algorithm outperforms the traditional or logarithmic EM algorithm in terms of convergence speed for an appropriate range of the design parameter α. The log-EM algorithm is a special case corresponding to α=-1. The main idea behind the α-EM algorithm is to search for an effective surrogate function or a minorizer for the maximization of the observed data´s likelihood ratio. The surrogate function adopted in this paper is based upon the α-logarithm which is related to the convex divergence. The convergence speed of the α-EM algorithm is theoretically analyzed through α-dependent update matrices and illustrated by numerical simulations. Finally, general guidelines for using the α-logarithmic methods are given. The choice of alternative surrogate functions is also discussed.
Keywords :
convergence of numerical methods; information theory; iterative methods; optimisation; α-EM algorithm; α-dependent update matrices; α-expectation-maximization algorithm; α-logarithmic information measures; CPU time; Gaussian mixtures; convergence speed; convex divergence; data likelihood ratio; design parameter; iteration counts; likelihood maximization algorithm; log-EM algorithm; logarithmic EM algorithm; numerical simulations; surrogate function; surrogate likelihood maximization; Algorithm design and analysis; Convergence of numerical methods; Guidelines; Helium; Independent component analysis; Instruction sets; Iterative algorithms; Numerical simulation; Unsupervised learning; Vector quantization;
Journal_Title :
Information Theory, IEEE Transactions on
DOI :
10.1109/TIT.2002.808105