Partial likelihood methods for probability density estimation

Partial likelihood (PL) establishes a sufficiently general framework to develop and study statistical properties of nonlinear techniques in signal processing. Adah et al. (1997), present the theorem by which the fundamental information-theoretic relationship for learning the PL cost, the equivalence of likelihood maximisation and relative entropy minimization, is established. In this paper, we reformulate the theorem to incorporate both the continuous and discrete probability modeling. We further show that, in both cases, the two conditions of the theorem are satisfied for the basic class of probability models, the exponential family, which includes many important network structures that can be effectively used as probability models. Hence we provide the prospect of using the PL cost in a wide class of applications with different models. We also propose several algorithms for learning/estimating the optimal model parameters by PL maximization. We give examples to illustrate the application of our general formulation and the learning algorithms and demonstrate the advantages of learning on the PL cost by simulation results.