Design of Natural Classification Kernels Using Prior Knowledge

A new class of kernels has been designed to enhance the usability of prior knowledge. Prior knowledge is shown to improve the generalization ability of kernel algorithms for binary classification problems. The prior knowledge is expressed in natural language via fuzzy rules. First, the concepts of a fuzzy rule base and prior-confidence region are proposed to formulate the prior knowledge. Then, the new kernels, which are referred to as natural classification kernels (NCKs), are represented by fuzzy equivalence relations based on the formulation of prior knowledge. An NCK is interpreted as a measure of similarities between samples. It is proven that NCKs have two desired properties: 1) transitivity with respect to the triangular norms and 2) the ability to provide higher similarities to spatially closer samples from the same class. Using transitivity, a large number of NCKs may be directly obtained by means of triangular norms. Additionally, the theoretical results show that the second property makes it possible for the support vector machine (SVM) and convex hull separation algorithm to generalize from training samples to test samples in the prior-confidence region. Finally, some synthetic datasets and a benchmark dataset are employed to validate the proposed approach.