An information-theoretic analysis of distributed resource allocation

Solving a resource allocation problem in a distributed way requires communication between the system and its users. This information exchange is, however, limited by communication constraints, delays, and distortions in most practical problems. This paper presents a quantitative analysis of information (flow) in a well-known distributed resource allocation algorithm using concepts from Shannon information theory. For this purpose, an entropy-based measure is adopted to quantify information which is defined as uncertainty reduction. Then, information flow in a certain class of iterative algorithms is studied. The relationships between the rate and total amount of information exchanged, and convergence of the algorithm are investigated under certain assumptions. The concepts introduced and the obtained results are illustrated using numerical examples.