Multilevel Coding for Nonlinear ISI Channels

An optimal equalizer for the transmission of densely modulated signals on nonlinear ISI channels requires the use of full-branched BCJR algorithm, which has an extremely high complexity. This paper proposes a new nonlinear equalization approach that significantly lowers the complexity and yields good performance at the same time. Multilevel coding is performed at the transmitter, and multistage equalization and decoding at the receiver. A linear complexity equalization scheme using nonlinear ISI cancellation is presented, as well as an equalization scheme based on the reduced-state BCJR algorithm for improved performance at a slightly higher complexity. Capacity analysis and code simulation results show that these two schemes substantially outperform the conventional approach, which views the nonlinear ISI as additive noise