Optimum Pulse Shapes for Pulse-Amplitude Modulation Data Transmission Using Vestigial Sideband Modulation

When designing pulse-amplitude modulation (PAM) systems for use over bandpass channels having restricted bandwidth, one may be led to use single-sideband (SSB) or vestigial sideband (VSB) transmission in order to maximize transmission rate. Carefully shaped pulses must be used to minimize degradation due to intersymbol interference (isi). In this paper VSB pulses are derived that have no inherent isi, and in addition have maximum immunity in a mean-squared-error (MSE) sense to timing and carrier phase errors. The results here generalize on those of Franks who treated the SSB case. The optimum VSB pulses are shown to be identical to those of Franks at the band edges, and to have vestigial roll-off characteristics of a similar discontinuous nature. It is also shown that one may allocate available bandwidth to the vestigial roll-off band band and Nyquist roll-off band in any way desired. The final performance for the optimum pulse depends only on the total bandwidth used. These results are shown to apply as well to a system employing a matched filter. The case of random timing and phase errors is also considered, and optimum pulses are again found. Several mathematical properties of the optimal pulses are given physical justification and geometric insight through error ellipses, and optimal signal shapes are plotted for various cases of special interest.