Support vector machine for functional data classification

In many applications, input data are sampled functions taking their values in infinite dimensional spaces rather than standard vectors. This fact has complex consequences on data analysis algorithms that motivate modifications of them. In fact most of the traditional data analysis tools for regression, classification and clustering have been adapted to functional inputs under the general name of functional Data Analysis (FDA). In this paper, we investigate the use of Support Vector Machines (SVMs) for functional data analysis and we focus on the problem of curves discrimination. SVMs are large margin classifier tools based on implicit non linear mappings of the considered data into high dimensional spaces thanks to kernels. We show how to define simple kernels that take into account the unctional nature of the data and lead to consistent classification. Experiments conducted on real world data emphasize the benefit of taking into account some functional aspects of the problems.

💡 Research Summary

This paper addresses the problem of classifying functional data—observations that are naturally represented as continuous functions rather than finite‑dimensional vectors—by extending Support Vector Machines (SVMs) to this setting. The authors begin by noting that functional data live in infinite‑dimensional Hilbert spaces, most commonly (L^{2}(\mu)), and that traditional FDA techniques rely on either dimensionality reduction (e.g., functional principal component analysis) or regularization (e.g., smoothing constraints) to avoid ill‑posedness. However, non‑linear classification methods have received comparatively little attention.

The core contribution is a systematic framework for applying SVMs directly to functional inputs. By recognizing that the dual formulation of SVM depends only on inner products between data points, the authors replace the Euclidean inner product with the (L^{2}) inner product (\langle x_i, x_j\rangle = \int x_i(t)x_j(t),d\mu(t)). This observation allows the use of kernel methods without explicitly mapping functions into a finite‑dimensional feature space.

Three families of kernels are proposed. (1) Standard kernels extended to functional spaces: the Gaussian (RBF) kernel (K(u,v)=\exp(-\sigma|u-v|^{2})) and polynomial kernel (K(u,v)=(1+\langle u,v\rangle)^{d}) are defined using the (L^{2}) norm and inner product. (2) Function‑specific kernels that incorporate derivative or integral operators, e.g., (K(u,v)=\int u’(t)v’(t),dt) or double‑integral kernels (\int!!\int u(s)v(t)\kappa(s,t),ds,dt). (3) Similarity‑based kernels based on functional distances or the Cauchy–Schwarz similarity. All kernels satisfy symmetry and positive‑definiteness, guaranteeing—by the Moore‑Aronszajn theorem—the existence of a reproducing kernel Hilbert space (RKHS) and an associated feature map (\phi).

Theoretical analysis focuses on consistency. Under mild assumptions (finite second moments of the functional inputs, continuity and positive‑definiteness of the kernel, and an appropriate scaling of the soft‑margin parameter (C) with the sample size (N)), the authors prove that the soft‑margin functional SVM converges to the Bayes optimal classifier as (N\to\infty). The proof leverages the equivalence between the primal soft‑margin problem and a regularized empirical risk minimization with the hinge loss, showing that the RKHS norm regularization induced by the kernel controls model complexity even in infinite dimensions.

Empirical validation is performed on three real‑world datasets: (a) speech recognition where each sample is a time‑frequency spectrogram curve, (b) chemical spectroscopy where absorbance is recorded over wavelengths, and (c) meteorological time series of temperature. For each dataset the authors compare (i) classical vector‑based SVMs (linear and RBF), (ii) SVM after functional PCA dimensionality reduction, and (iii) the proposed functional‑kernel SVMs. Results consistently show that functional kernels achieve higher accuracy, precision, and recall. Notably, kernels that embed derivative information are especially robust to noise, and the functional kernels exhibit lower sensitivity to hyper‑parameter choices than their vector counterparts.

In summary, the paper delivers a mathematically rigorous yet practically viable extension of SVMs to functional data. It demonstrates that by designing kernels that respect the functional nature of the inputs, one can retain the large‑margin benefits of SVMs while avoiding the pitfalls of high‑dimensional linear classifiers. The work opens several avenues for future research, including multi‑output functional data (e.g., image sequences), integration with deep learning feature extractors, automated kernel learning for functional spaces, and scalable approximations for very large functional datasets.