New Convolutive-Error-Measure and Minimum Total-Model-Order Determination Algorithm for Equalization in Communications

The inverse system approximation using the finite impulse responses (FIR) and the corresponding total-model-order determination are essential to a broad area of signal processing, telecommunication, control applications such as acoustic echo cancellation, communication equalization, plant control, etc. To the best of our knowledge, there exists no explicit formulation of the exact Li convolutive error for any arbitrary filter (system) and the corresponding truncated inverse filter. In addition, the approach to determine the minimum total-model-order of the inverse filter subject to the maximum allowable Li convolutive error is also in demand. In this paper, we first derive the formula of the Li convolutive error measure with respect to the filter coefficients for any arbitrary system. According to this new error measure, we design an optimal inverse FIR filter given the exact model orders to achieve the minimum convolutive error. Then, we propose a new algorithm to determine the minimum total-model-order of the appropriate truncated inverse filter to achieve a specified convolutive error based on the discrete filled function approach. A new tradeoff objective function can therefore be facilitated. The numerical evaluation is demonstrated for such a tradeoff between the total-model-order and the system performance, e.g., the bit error rate (BER) for a communication receiver compensated by an FIR equalizer.