Extension of the Blahut-Arimoto algorithm for maximizing directed information

We extend the Blahut-Arimoto algorithm for maximizing Massey´s directed information, which can be used for estimating the capacity of channels with delayed feedback. In order to do so, we apply the ideas from the regular Blahut-Arimoto algorithm, i.e., the alternating maximization procedure, onto our new problem, and show its convergence to the global optimum value. Our main insight in this paper is that in order to find the maximum of the directed information over causal conditioning probability mass function, one can use a backward index time maximization combined with the alternating maximization procedure. We give a detailed description of the algorithm, its complexity, and memory needed. Some numerical results are provided to illustrate the performance of the suggested algorithm.