On the optimal weight vector of a perceptron with Gaussian data and arbitrary nonlinearity

The authors investigate the solution to the following problem: find the optimal weighted sum of given signals when the optimality criterion is the expected value of a function of this sum and a given `training´ signal. The optimality criterion can be a nonlinear function from a very large family of possible functions. A number of interesting cases fall under this general framework, such as a single layer perceptron with any of the commonly used nonlinearities, the least-mean-square (LMS), the LMF or higher moments, or the various sign algorithms. Assuming the signals to be jointly Gaussian, it is shown that the optimal solution, when it exits, is always collinear with the well-known Wiener solution, and only its scaling factor depends on the particular functions chosen. Necessary constructive conditions for the existence of the optimal solution are also presented