Hierarchical modulation-based cooperative scheme: Minimizing the symbol error probability

In a cooperative network, relays are often closer to the source than the destination and thus the source-relay links have a higher capacity than the source-destination one. In this paper, we investigate a hierarchical modulation-based cooperative scheme, where the relay is able to detect the whole signal, while the destination can only detect the □rst layer of the modulation. We provide an error probability upper-bound for this scheme, which allows us to optimize the choice of the constellation parameter θ.