Optimal signaling and detector design for power constrained on-off keying systems in Neyman-Pearson framework

Optimal stochastic signaling and detector design are studied for power constrained on-off keying systems in the presence of additive multimodal channel noise under the Neyman-Pearson (NP) framework. The problem of jointly designing the signaling scheme and the decision rule is addressed in order to maximize the probability of detection without violating the constraints on the probability of false alarm and the average transmit power. Based on a theoretical analysis, it is shown that the optimal solution can be obtained by employing randomization between at most two signal values for the on-signal (symbol 1) and using the corresponding NP-type likelihood ratio test at the receiver. As a result, the optimal parameters can be computed over a significantly reduced optimization space instead of an infinite set of functions using global optimization techniques. Finally, a detection example is provided to illustrate how stochastic signaling can help improve detection performance over various optimal and sub-optimal signaling schemes.