Performance of the adaptive baseband 4D diagonalizer, a crosstalk canceller, with four-dimensional constellations

A method of calculating the probability of symbol error for the four-dimensional (4-D) diagonalizer using arbitrary constellations is presented. The method is applied to a 4-D constellation introduced by A. Gersho and V.B. Lawrence (1984), which is called the GL4-256 constellation. The probability of symbol error performance for GL4-256 is compared to that for the dual-channel 16-QAM case, and it is concluded that GL4-256 performs better for low crosstalk values, i.e., below -9.5 dB, with, the reverse being true for high crosstalk levels. Graphical results are presented for the case of uniform distribution