On batch learning in a binary weight setting

Considers the problem of inferring a finite binary sequence w*∈{-1,1}ⁿ from a random sequence of half-space data {u ^(t)∈{-1,1}ⁿ:⟨w*,u/sup (t/)⟩⩾0,t⩾1}. In this context, we show that a previously proposed randomised on-line learning algorithm dubbed directed drift [Venkatesh, 1993] has minimal space complexity but an expected mistake bound exponential in n. We show that batch incarnations of the algorithm allow of massive improvements in running time. In particular, using a batch of ½πn log n examples at each update epoch reduces the expected mistake bound to 𝒪(n) in a single bit update mode, while using a batch of πn log n examples at each update epoch in a multiple bit update mode leads to convergence to w* with a constant (independent of n) expected mistake bound