Joint Optimization of the Transmit Covariance and Relay Precoder in General Gaussian Amplify-and-Forward Relay Channels

The maximum data rate that can be achieved by the strictly causal full-duplex amplify-and-forward (AF) scheme in general Gaussian relay channels is achieved by Gaussian codebooks and can be cast as the solution of an optimization problem of the input transmit covariance and relay precoder. This problem possesses an intricate nonconvex structure and is hence difficult to solve. To circumvent this difficulty, the relay precoder is assumed to be given and then the Karush-Kuhn-Tucker conditions are used to obtain closed form expressions for the optimal input covariance corresponding to that precoder. These expressions are used to show that subdiagonal precoders suffice to attain the maximum achievable rate of the AF scheme at any source transmit power. In addition to significantly reducing the effort expended in searching for the optimal relay precoder, this observation enables us to find the optimal precoders at low and high source transmit powers. For asymptotically low transmit powers, the optimal relaying mechanism is shown to possess an interlacing structure, thereby resembling half-duplex operation. In contrast, for asymptotically high transmit powers, it is optimal for the relay to be silent. The asymptotic analysis enables us to develop an explicit formulation for a suboptimal precoder that, at intermediate source transmit powers, are shown numerically to outperform asymptotically optimal precoders.