Abstract :
In this paper, the design of physically realizable rational fuction transmitting or receiving filters for use in pulse transmission systems operating in the presence of Gaussian noise and intersymbol interference is explored. For the design, the three iteria considered are 1) mean-square error (MSE), 2) error probability, and 3) a weighted sum of the squares of the signal-to-noise ratios corresponding to all possible received signal patterns (MSSN). Expressions are obtained for the various error criteria in terms of the transnmission system poles and residues (coefficients of a partial fraction expansion), assuming that the transmitting and receiving filters and the transmission medium are given by physically realizable rational function forms. It is shown that optimization of the MSE criterion under a received signal amplitude constraint with respect to the receiving filter residues, for a fixed set of poles, leads to a set of linear equations readily solvable for the optimal residues. A suboptimal technique is used to specify "reasonable" pole values, thereby the poles are constrained to belong to some "standard" set of all-pole transmission functions, as for example, maximally flat delay or maximally flat magnitude. The bandwidth of the given pole configuration is determined to optimize the given error criterion. Numemical examples are presented to illustrate the filter design techniques developed. The results indicate that, in many cases, filter design under the MSE or MSSN error criteria leads to optimal or near optimal design under an error-probability criterion. A brief discussion is also given of the filter sensitivity to parameter variations.
