Title :
Information theoretic upper bounds on the number of distinguishable classes
Author :
Keller, Catherine M. ; Ho, Mantak ; Basu, Prithwish ; Whipple, Gary H.
Author_Institution :
MIT Lincoln Lab., Lexington, MA, USA
Abstract :
This paper examines data driven information theoretic upper bounds on the number of classes that can be distinguished by machine-learning classification systems as a function of the signal-to-noise ratio (SNR) of the features. Fano upper bounds are derived with desired classification error as a parameter. A simulation example is used to explore the bounds.
Keywords :
learning (artificial intelligence); signal classification; Fano upper bounds; SNR; classification error; distinguishable classes; information theoretic upper bounds; machine-learning classification systems; signal-to-noise ratio; Covariance matrices; RLC circuits; Random variables; Signal to noise ratio; Training; Training data; Upper bound;
Conference_Titel :
Signals, Systems and Computers, 2013 Asilomar Conference on
Conference_Location :
Pacific Grove, CA
Print_ISBN :
978-1-4799-2388-5
DOI :
10.1109/ACSSC.2013.6810500
