Joint optimization of the transmit covariance and the relay precoder in general Gaussian amplify-and-forward relay channels

The capacity of the amplify-and-forward (AF) scheme in general full-duplex Gaussian relay channels is achieved by Gaussian codebooks and can be cast as the solution of an optimization problem of the input transmit covariance and the relay precoder. This problem is non-convex. To circumvent this difficulty, the Karush-Kuhn-Tucker (KKT) conditions are used to obtain closed form expressions of the optimal input covariance that corresponds to an arbitrary relay precoder. Using these expressions, it is shown the maximum rate of the AF scheme is achieved by subdiagonal precoders. This observation is used to facilitate the search for the optimal relay precoder, and to show that at high transmit powers, it is optimal for the relay to remain silent and, at low transmit powers, it is optimal to operate in a mode that resembles half-duplex operation.