Information theory and neuroscience: Why is the intersection so small?

Information theory sprung from Shannonpsilas desire to determine the performance limits of communication systems. Neuroscience seeks to determine how an existing system the brain encodes and processes information. Information theory has made few inroads into neuroscience and when it has, the theory has occasionally been applied incorrectly in both obvious and subtle ways. We review both successes and failures of classic information theory in neural coding studies. We present a non-classical approach to the analysis motivated by neuroscience problems that relies heavily on the data processing theorem and properties of the Kullback-Leibler distance.