Universal Source Coding for Monotonic and Fast Decaying Monotonic Distributions

We study universal compression for sequences generated by monotonic distributions. We show that for a monotonic distribution over an alphabet of size k, each probability parameter costs essentially 0.5 log (n/k³) bits, where n is the coded sequence length, as long as k = o(n^1/3). Otherwise, for k = O(n), the total average sequence redundancy is O(n^1/3+epsiv) bits overall. We then show that there exists a sub-class of monotonic distributions over infinite alphabets for which redundancy of O(n^1/3+epsiv) bits overall is still achievable. This class contains fast decaying distributions, including distributions over the integers and geometric distributions. For some slower decays, redundancy of o(n) bits overall is achievable.