Optimal supervised learning with two teachers

We study Hebbian supervised learning of a linearly separable function in the presence of two instructing perceptrons: The teacher (B) that carries the correct function, or rule to be learned, and the assistant teacher (T) that is itself in the process of learning the rule. In the assistant teacher’s training set only the teacher perceptron is used. In the student’s training set, on the other hand, the answers are linear combinations of the answers coming from the teacher and the answers coming from the assistant teacher σB+aσT. We demonstrate through analytical results that the generalization gain per example can be improved beyond the typical Hebbian α−1/2 dependence by adjusting the attention parameter “a”. The optimal result, that corresponds to the case where , and the assistant teacher and the student have the same training questions, gives us generalization error decaying with α−3/4.