On Pinsker\´s and Vajda\´s Type Inequalities for Csiszár\´s $f$ -Divergences

Let D and V denote respectively Information Divergence and Total Variation Distance. Pinsker´s and Vajda´s inequalities are respectively D ≥ [ 1/ 2] V² and D ≥ log[( 2+V)/( 2-V)] - [( 2V)/( 2+V)]. In this paper, several generalizations and improvements of these inequalities are established for wide classes of <;i>f<;/i>-divergences. First, conditions on f are determined under which an f-divergence Df will satisfy Df ≥ cf V² or Df ≥ c_2,f V² + c_4,f V⁴, where the constants cf, c_2,f and c_4,f are best possible. As a consequence, lower bounds in terms of V are obtained for many well known distance and divergence measures, including the χ² and Hellinger´s discrimination and the families of Tsallis´ and Rényi´s divergences. For instance, if D_(α) (P||Q) = [α(α-1)]^-1 [∫p^αq^1-αdμ-1] and ℑ_α (P||Q) = (α-1)^-1 log[∫p^αq^1-αdμ] are respectively the relative information of type α and the Rényi´s information gain of order α, it is shown that D_(α) ≥ [ 1/ 2] V² + [ 1/ 72] (α+1)(2-α) V⁴ whenever -1 ≤ α ≤ 2, α ≠ 0,1 and that ℑ_α ≥ [( α)/ 2] V² + [ 1/ 36] α(1 + 5 α- 5 α² ) V⁴ for 0 <; α <; 1. In a somewhat - - different direction, and motivated by the fact that these Pinsker´s type lower bounds are accurate only for small variation (V close to zero), lower bounds for Df which are accurate for both small and large variation (V close to two) are also obtained. In the special case of the information divergence they imply that D ≥ log[ 2/( 2-V)] - [( 2-V)/2] log[( 2+V)/2], which uniformly improves Vajda´s inequality.