DrSVM: Distributed random projection algorithms for SVMs

We present distributed random projected gradient algorithms for Support Vector Machines (SVMs) that can be used by multiple agents connected over a time-varying network. The goal is for the agents to cooperatively find the same maximum margin hyperplane. In the primal SVM formulation, the objective function can be represented as a sum of convex functions and the constraint set is an intersection of multiple halfspaces. Each agent minimizes a local objective subject to a local constraint set. It maintains its own estimate sequence and communicates with its neighbors. More specifically, each agent calculates weighted averages of the received estimates and its own estimate, adjust the estimate by using gradient information of its local objective function and project onto a subset of its local constraint set. At each iteration, an agent considers only one halfspace since projection onto a single halfspace is easy. We also consider the convergence behavior of the algorithms and prove that all the estimates of agents converge to the same limit point in the optimal set.