Feature kernel functions: improving SVMs using high-level knowledge

Kernel functions are often cited as a mechanism to encode prior knowledge of a learning task. But it can be difficult to capture prior knowledge effectively. For example, we know that image pixels of a handwritten character result from a few strokes from a single writing implement; it is not clear how to express this in a kernel function. We investigate an explanation based learning (EBL) paradigm to generate specialized kernel functions. These embody novel high-level features that are automatically constructed from the interaction of prior knowledge and training examples. Our empirical results showed that the performance of the resulting SVM surpasses that of a conventional SVM on the challenging task of classifying handwritten Chinese characters.