On the complexity of hypothesis space and the sample complexity for machine learning

Author

Nakazawa, Makoto ; Kohnosu, Toshiyuki ; Matsushima, Toshiyasu ; Hirasawa, Shigeichi

Author_Institution

Dept. of Ind. Eng. & Manage., Waseda Univ., Tokyo, Japan

Volume

1

fYear

1994

fDate

2-5 Oct 1994

Firstpage

132

Abstract

The problem of learning a concept from examples in the model introduced by Valiant (1984) is discussed. According to the traditional ways of thinking, it is assumed that the learnability is independent of the occurrence probability of instance. By utilizing this probability, we propose the metric as a new measure to determine the complexity of hypothesis space. The metric measures the hardness of discrimination between hypotheses. Furthermore, we obtain the average metric dependent on prior information. This metric is the measure of complexity for hypothesis space in the average. Similarly in the worst case, we obtain the minimum metric. We make clear the relationship between these measures and the Vapnik-Chervonenkis (VC) dimension. Finally, we show the upper bound on sample complexity utilizing the metric. This results can be applied in the discussion on the learnability of the class with an infinite VC dimension

Keywords

computational complexity; learning (artificial intelligence); learning systems; probability; PAC learning model; Vapnik-Chervonenkis dimension; hypothesis space complexity; infinite VC dimension; learnability; machine learning; minimum metric; occurrence probability; sample complexity; upper bound; Engineering management; Extraterrestrial measurements; Industrial engineering; Machine learning; Telecommunications; Upper bound; Virtual colonoscopy;

fLanguage

English

Publisher

ieee

Conference_Titel

Systems, Man, and Cybernetics, 1994. Humans, Information and Technology., 1994 IEEE International Conference on

Conference_Location

San Antonio, TX

Print_ISBN

0-7803-2129-4

Type

conf

DOI

10.1109/ICSMC.1994.399824

Filename

399824