Optimum transmitting filter in digital PAM systems with a Viterbi detector

Optimization of the transmitting filter in a PAM system using a Viterbi detector of constrained complexity is considered. The receiving filter is considered to be a whitened matched filter. A constraint on detector complexity is obtained by limiting the length of the system impulse response. The results are applied to a channel with coaxial cable characteristics. Comparison with other detectors shows that the Viterbi detector is preferable even when the length of the system impulse response is quite short.