Lasso type classifiers with a reject option

We consider the problem of binary classification where one can, for a particular cost, choose not to classify an observation. We present a simple proof for the oracle inequality for the excess risk of structural risk minimizers using a lasso type penalty.

💡 Research Summary

The paper addresses binary classification when a “reject” decision is allowed at a prescribed cost d (0 ≤ d ≤ ½). Instead of the usual 0‑1 loss, the authors define a piece‑wise loss ℓ(z) that penalises a wrong classification with cost 1, a rejection with cost d, and gives zero loss otherwise. A threshold τ (0 ≤ τ < 1) determines when the classifier with discriminant function f(x) makes a decision (|f(x)| > τ) or rejects (|f(x)| ≤ τ). The Bayes optimal rule f₀(x) is explicitly given: it outputs –1, 0, +1 depending on whether the conditional class probability η(x)=P(Y=+1|X=x) lies below d, between d and 1–d, or above 1–d.

To estimate f₀, the authors consider a finite dictionary of base functions F_M={f₁,…,f_M} and look for a linear combination f_λ(x)=∑{j=1}^M λ_j f_j(x). They introduce a convex surrogate loss φ (Lipschitz continuous) and define the empirical risk R̂_φ(λ) = (1/n)∑{i=1}^n φ(Y_i f_λ(X_i)). Regularisation is performed with an ℓ₁‑penalty p(λ)=2 r_n‖λ‖₁, leading to the penalised empirical risk minimisation problem
  λ̂ = arg min_{λ}