A novel entropy estimator and its application to ICA

We present a new (differential) entropy estimator where the maximum entropy bound is used to approximate the entropy given the observations, and is computed using a numerical procedure. The resulting accurate estimate for the entropy is used to derive a new algorithm to perform independent component analysis (ICA). The new algorithm, ICA by entropy bound minimization (ICA-EBM), adopts a line search procedure, and initially uses updates that constrain the demixing matrix to be orthogonal for robust performance. We present simulation results that demonstrate the superior performance of ICA-EBM and its ability to match sources that come from a wide range of distributions.