Invariant Coordinate Selection and Fisher discriminant subspace beyond the case of two groups

Invariant Coordinate Selection (ICS) is a multivariate technique that relies on the simultaneous diagonalization of two scatter matrices. It serves various purposes, including its use as a dimension reduction tool prior to clustering or outlier detection. ICS’s theoretical foundation establishes why and when the identified subspace should contain relevant information by demonstrating its connection with the Fisher discriminant subspace (FDS). These general results have been examined in detail primarily for specific scatter combinations within a two-cluster framework. In this study, we expand these investigations to include more clusters and scatter combinations. Our analysis reveals the importance of distinguishing whether the group centers matrix has full rank. In the full-rank case, we establish deeper connections between ICS and FDS. We provide a detailed study of these relationships for three clusters when the group centers matrix has full rank and when it does not. Based on these expanded theoretical insights and supported by numerical studies, we conclude that ICS is indeed suitable for recovering the FDS under very general settings and cases of failure seem rare.

💡 Research Summary

This paper extends the theoretical understanding of Invariant Coordinate Selection (ICS) as a tool for recovering the Fisher Discriminant Subspace (FDS) beyond the traditional two‑group setting. The authors consider mixture models of the form

 fY(y)=|Γ|⁻¹/2 ∑_{j=1}^k α_j g_j((y−μ_j)ᵀΓ⁻¹(y−μ_j)),

where k≥2 groups have distinct means μ_j, common within‑group scatter Γ, and mixing proportions α_j. Building on Tyler et al. (2009), they recall that for any pair of affine‑equivariant scatter matrices V₁ and V₂, the generalized eigenvalues of V₁⁻¹V₂ contain at least one eigenvalue ρ* whose multiplicity is ≥ p−q, where p is the data dimension and q=rank(M) with M=