Optimal Volterra kernel estimation algorithms for a nonlinear communication system for PSK and QAM inputs

A fifth-order Volterra kernel estimation algorithm, which is optimal in the least mean square error sense, for a bandpass nonlinear system is derived. The algorithm is based on some characteristics of i.i.d. circularly symmetric zero-mean complex-valued Gaussian random variables. The proposed algorithm can be used to identify a nonlinear system under uniformly i.i.d. rectangular M-QAM input and under uniformly i.i.d. M-PSK input (M⩾4) with modest modification. The same approach has been used to derive an optimal Volterra kernel estimation algorithm up to the third order. However, in some cases, a third-order model is not of “high enough order” to capture the nonlinear system characteristics. A simulation example is given to show the necessity of deriving a fifth-order Volterra kernel estimation algorithm and to test for the correctness of the algorithm