Computational power of neural networks: a characterization in terms of Kolmogorov complexity

Author

Balcázar, José L. ; Gavaldà, Ricard ; Siegelmann, Hava T.

Author_Institution

Dept. of Software, Univ. Politecnica de Catalunya, Barcelona, Spain

Volume

43

Issue

4

fYear

1997

fDate

7/1/1997 12:00:00 AM

Firstpage

1175

Lastpage

1183

Abstract

The computational power of recurrent neural networks is shown to depend ultimately on the complexity of the real constants (weights) of the network. The complexity, or information contents, of the weights is measured by a variant of resource-bounded Kolmogorov (1965) complexity, taking into account the time required for constructing the numbers. In particular, we reveal a full and proper hierarchy of nonuniform complexity classes associated with networks having weights of increasing Kolmogorov complexity

Keywords

Turing machines; computational complexity; information theory; recurrent neural nets; Kolmogorov complexity; Turing machines; computational power; hierarchy theorem; information contents; information theory; nonuniform complexity classes; real constants; recurrent neural networks; resource bounded Kolmogorov complexity; weight complexity; Analog computers; Computer networks; Feedback loop; Intelligent networks; Neural networks; Recurrent neural networks; Time factors; Time measurement; Turing machines; Weight measurement;

fLanguage

English

Journal_Title

Information Theory, IEEE Transactions on

Publisher

ieee

ISSN

0018-9448

Type

jour

DOI

10.1109/18.605580

Filename

605580