Performance analysis of an RLS-LMS algorithm for lossless audio compression

We analyze the effect of the input signal correlation on the performance of an RLS-LMS based adaptive linear prediction algorithm for lossless audio compression. We show that the misadjustment is the same for both correlated and uncorrelated input samples. However, the variance of the residual error power of the algorithm increases as the signal variance increases, which may cause numerical instability or a stalling problem for the RLS algorithm. For real audio signals quantized at 16 bit and 24 bit, experimental results show that the algorithm is capable of modelling audio signals quantized at 16 bit, but it can yield reduced performance coding gain for audio signals quantized at 24 bit, which confirms our theoretical analysis.