Transfer learning in a heterogeneous environment

We present a method for transfer learning, in which tasks encountered in the past are used to choose a representation which is expected to work well on future tasks. Each task is assumed to be binary classification or regression in a Hilbert space. We propose to arrange the observed tasks into groups and to assign a low-dimensional projection to each group. The groups and the corresponding projections are chosen to minimize an empirical error criterion. To learn a future task, one selects the projection, and the corresponding linear function, for which the empirical error is minimal. The expected error of this method when applied to a future task is shown to be uniformly bounded by the empirical error criterion. The bound is independent of the dimension of the Hilbert space. The advantages of transfer learning over single task learning and the advantages of task grouping over no grouping are discussed.