MSE-based source and relay precoder design for cognitive radio multiuser two-way relay systems

We consider joint design of source and relay pre-coders in a cognitive multiuser two-way relay system, which supports simultaneous transmission of multiple secondary users concurrently with primary network with the help of a relay node. The design criterion is to minimize the sum mean square error (MSE) of all users under a transmit power constraint for each transmitting node while causing no interference to the primary network. To solve this non-convex optimization problem, an iterative algorithm is proposed to iteratively solve for the precoding matrices at secondary source nodes and relay node, and then the decoding matrices at secondary nodes. To reduce the computational complexity, a matrix distance based non-iterative algorithm is also proposed. Simulation results show the effectiveness of our proposed algorithms.