Penalty and margin decomposition - an inspection of loss function regularization in SVM

In general, a classifier of statistical learning can be expressed as a regularized optimization problem argmin_f∈H λ_ΩΩ[f]+R_emp[f] where λ_Ω is a regulator for balancing the optimization between Ω[f] and R_emp[f] terms. The λ_Ω here is a factor for regularization over all the training patterns. By the regulator λ_Ω, the optimization weights all the training patterns uniquely with an equivalent cost despite there are different altitudes among the training samples. The altitudes may vary with the difference in sampling cost, the different uncertainties behind the samples, or even the imbalance between the adversary classes. Models capable of assigning various costs for individual samples are hence developed in this paper. By passing the regularization to R_emp[f] term, this study proposes different regularization models of the support vector machines by tuning a parameterized governing loss function. Since the loss function is a key for success of the support vector machines, changing the loss function individually extends the support vector machines capable of accomplishing the missions mentioned above. This study discovers the properties due to the changes in loss function, and realizes in turn the feasibility for three kinds of related models.