Sample complexity of salient feature identification for sparse signal processing

Suppose among a set of N covariates X₁, X₂, ..., X_N there is a subset S of covariates that are salient for predicting outcomes Y. Specifically, we assume that when Y is conditioned on {Y_k}_k∈S it is independent of the other covariates. Our goal is to identify the subset S from data samples of the covariates and the associated outcomes. We first consider the case where the covariates are independent of each other and then generalize the results to the case where the covariates are dependent with symmetric distributions. We present precise mutual information expressions that characterize the sample complexity for accurately identifying the subset S. We then derive sample complexity bounds for interesting scenarios.