Signal Processing Using Mutual Information

The Darbellay-Vajda algorithm was used to develop a skeletonized approximation to a joint probability density of sampled data. The approximation is presented as a collection of non-overlapping multi-dimensional cuboids, having varying sizes, locations, and probabilities in sample space. It is already known that a mutual information value can be extracted from this collection. This paper demonstrates that the joint density has a far wider range of application in exploring Bayesian and conditional probability distributions among the observations. While the examples provided show only autonomous data modeling, categorical data is easily input as an additional independent variable for supervised training purposes. Though the mathematical fundamentals of the algorithm are hardly straightforward, the associated computation load is low, and the overall flexibility of the technique points to the possibility of attractive new algorithms for statistical signal processing in numerous areas such as machine learning, pattern recognition, and nonlinear filtering