Reconsidering unique information: Towards a multivariate information decomposition

The information that two random variables Y, Z contain about a third random variable X can have aspects of shared information (contained in both Y and Z), of complementary information (only available from (Y, Z) together) and of unique information (contained exclusively in either Y or Z). Here, we study measures SĨ of shared, UĨ unique and CĨ complementary information introduced by Bertschinger et al. [1] which are motivated from a decision theoretic perspective. We find that in most cases the intuitive rule that more variables contain more information applies, with the exception that SĨ and CĨ information are not monotone in the target variable X. Additionally, we show that it is not possible to extend the bivariate information decomposition into SĨ, UĨ and CĨ to a non-negative decomposition on the partial information lattice of Williams and Beer [2]. Nevertheless, the quantities UĨ, SĨ and CĨ have a well-defined interpretation, even in the multivariate setting.