Cooperative joint precoding for relay enhanced cellular system over Nakagami-m fading channels

In this paper, we introduce the notion of a relay enhanced cellular system, where a relay station is located in the overlap area served by two base stations. We propose a cooperative joint precoding scheme in the downlink of the relay enhanced cellular system to maximize the system capacity while eliminating the interference at the mobile stations. We formulate an optimization problem to maximize the system capacity and design the multiuser precoding vectors for each base station and the relay station. We analyze the ergodic rate performance of the proposed multiuser precoding scheme over Nakagami-m fading channels. Numerical results show that the proposed scheme can effectively cancel the interference and improve the sum rate and outage performance for cell edge users.