Learning with prior information

In this paper, a new notion of learnability is introduced, referred to as learnability with prior information (w.p.i.). This notion is weaker than the standard notion of PAC (probably approximately correct) learnability which has been much studied during recent years. A property called “dispersability” is introduced, and it is shown that dispersability plays a key role in the study of learnability w.p.i. Specifically, dispersability of a function class is always a sufficient condition for the function class to be learnable; moreover, in the case of concept classes, dispersability is also a necessary condition for learnability w.p.i. Thus in the case of learnability w.p.i., the dispersability property plays a role similar to the finite metric entropy condition in the case of PAC learnability with a fixed distribution. It is further shown in the paper that, if a function class consists of measurable functions mapping a separable metric space into a compact subset of R, then such a function class is automatically learnable w.p.i. In particular, any collection of measurable subsets of Rⁿ, for any integer n, is automatically learnable w.p.i. Next, the notion of learnability w.p.i. Is extended to the distribution-free situation, and it is shown that a property called d.f. dispersability (introduced here) is always a sufficient condition for d.f. learnability w.p.i., and is also a necessary condition for d.f. learnability in the case of concept classes