General treatment of the problem of coding

A typical communication system consists of the following five elements: (1) An information source. This can be considered to be represented mathematically by a suitable stochastic process which chooses one message from a set of possible messages. The rate R of producing information is measured by the entropy per symbol of the process. (2) An encoding or transmitting element. Mathematically this amounts to a transformation applied to the message to produce the signal, i.e., the encoded message. (3) A channel on which the signal is transmitted from transmitter to receiver. During transmission the signal may be perturbed by noise. (4) A receiving and decoding (or demodulating) device which recovers the original message from the received signal. (5) The destination of the information, e.g., the human ear (for telephony) or the eye (for television). The characteristics of the destination may determine the significant elements of the information to be transmitted. For example, with sound transmission, precise recovery of the phases of components is not required because of the insensitivity of the ear to this type of distortion.