Improving error probability of the prefiltered Viterbi equalizer

The article deals with the design of suboptimal data detectors for binary transmission over frequency selective channels. A detector consisting of a prefilter followed by a Viterbi processor with a number of states lower than that needed for maximum likelihood sequence estimation is considered. Often the parameters of the receiver are optimized according to the minimum mean square error criterion. Here a stochastic approximation algorithm that optimizes the parameters of the receiver according to a measure of the error probability is introduced. Simulation results show that the proposed design gives substantial benefits at moderate to high signal to noise ratio.