Blind source separation: are information maximization and redundancy minimization different?

Author

Obradovic, D. ; Deco, G.

Author_Institution

Central Technol. Dept., Siemens AG, Munich, Germany

fYear

1997

fDate

24-26 Sep 1997

Firstpage

416

Lastpage

425

Abstract

This paper provides a detailed and rigorous analysis of the two commonly used methods for blind source separation: linear independent component analysis (ICA) and information maximization (InfoMax). The paper shows analytically that ICA based on Kullback-Leibler information as a mutual information measure and InfoMax lead to the same solution if the parameterization of the output nonlinear functions in the latter method is sufficiently rich. Furthermore, this work discusses the alternative redundancy measures not based on the Kullback-Leibler information distance and nonlinear ICA. The practical issues of applying ICA and InfoMax are also discussed

Keywords

covariance matrices; feature extraction; maximum entropy methods; neural nets; probability; redundancy; Kullback-Leibler information; blind source separation; information maximization; linear independent component analysis; mutual information measure; redundancy minimization; Biological neural networks; Blind source separation; Covariance matrix; Feature extraction; Independent component analysis; Information analysis; Mutual information; Principal component analysis; Probability density function; Vectors;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks for Signal Processing [1997] VII. Proceedings of the 1997 IEEE Workshop

Conference_Location

Amelia Island, FL

ISSN

1089-3555

Print_ISBN

0-7803-4256-9

Type

conf

DOI

10.1109/NNSP.1997.622423

Filename

622423