On "Soft" bit allocation

If a random variable X with variance σ²is quantized optimally, being mapped into s discrete levels, the quantization error is roughly proportional to σ²/s². In many applications in speech coding and image digitization, we are given sets of random variables and we have to quantize them in some way so as to represent their realizations with as few discretized levels (bits) as possible. This has to be done while also minimizing the total quantization error. The quantization level or bit allocation process should therefore be the result of a compromise between the total discretization error and the number of bits used to represent the realizations of random vectors. This paper presents a solution of this problem, extending the classical bit allocation methods in which a fixed, prescribed number of bits have to be allocated to minimize the quantization error. The general soft bit allocation process is useful in designing variable rate (adaptive) coders as opposed to the classical bit allocation procedures that were devised for information transmission over communication channels requiring constant rate encoding.