OnDA: On-device Channel Pruning for Efficient Personalized Keyword Spotting

Always-on keyword spotting (KWS) demands on-device adaptation to cope with user- and environment-specific distribution shifts under tight latency and energy budgets. This paper proposes, for the first time, coupling weight adaptation (i.e., on-device training) with architectural adaptation, in the form of online structured channel pruning, for personalized on-device KWS. Starting from a state-of-the-art self-learning personalized KWS pipeline, we compare data-agnostic and data-aware pruning criteria applied on in-field pseudo-labelled user data. On the HeySnips and HeySnapdragon datasets, we achieve up to 9.63x model-size compression with respect to unpruned baselines at iso-task performance, measured as the accuracy at 0.5 false alarms per hour. When deploying our adaptation pipeline on a Jetson Orin Nano embedded GPU, we achieve up to 1.52x/1.57x and 1.64x/1.77x latency and energy-consumption improvements during online training/inference compared to weights-only adaptation.

💡 Research Summary

The paper addresses the challenge of adapting always‑on keyword spotting (KWS) systems to user‑ and environment‑specific distribution shifts while respecting the tight latency and energy constraints of edge devices. Existing on‑device personalization methods focus solely on weight fine‑tuning, which limits efficiency when the model architecture remains unchanged. To overcome this, the authors propose an “On‑Device Adaptation” (OnDA) framework that simultaneously adapts model weights and architecture through structured channel pruning performed on‑device.

The baseline pipeline (B1‑B3) follows the self‑learning approach of prior work: a ProtoNet‑style embedding network is pretrained on a large multi‑speaker dataset (MSWC) using a triplet loss; a few user‑provided positive examples are used to compute a keyword prototype; incoming audio is pseudo‑labeled by distance to this prototype; and finally the pseudo‑labeled data are used for on‑device triplet‑loss fine‑tuning.

OnDA extends this pipeline by inserting pruning operations at three possible moments: (P) offline pruning before any on‑device adaptation, (O1) online pruning before weight fine‑tuning, and (O2) online pruning after fine‑tuning. Two pruning criteria are evaluated: a data‑agnostic global L1‑norm magnitude pruning (cheap, suitable for O2) and a data‑aware Hessian‑Aware Pruning (HAP) that estimates the trace of the Hessian per channel via Hutchinson’s stochastic trace estimator (more expensive but captures loss curvature, suitable for O1).

Experiments are conducted on two public KWS datasets, HeySnips and HeySnapdragon, using two strong backbone architectures (ResNet15 and DS‑CNN‑L). Models are first pretrained, then optionally pruned offline at 25 % or 50 % ratios, and finally subjected to online pruning at 25 %, 50 % or 75 % ratios. The authors report Pareto fronts of accuracy (measured at a false‑alarm rate of 0.5 per hour) versus model size, as well as latency and energy measurements on an NVIDIA Jetson Orin Nano (GPU and CPU).

Key findings include:

1. Model‑size vs accuracy – OnDA pipelines consistently dominate the baseline and offline‑pruned models. For HeySnips, the best OnDA configuration achieves a 3.33× compression relative to the ResNet15 baseline and a 9.63× compression relative to the DS‑CNN‑L baseline while maintaining identical accuracy. For HeySnapdragon, the best compression is 1.7× over ResNet15.

2. Pruning criterion – Data‑aware HAP applied before fine‑tuning (O1) yields higher accuracy for a given compression than data‑agnostic L1 pruning applied after fine‑tuning (O2). The curvature‑aware scores better preserve channels critical for the in‑field distribution, even when only a few pseudo‑labeled samples are available.

3. Pruning timing – Performing pruning before weight adaptation (O1) reduces the total amount of computation required for subsequent fine‑tuning, whereas pruning after adaptation (O2) necessitates an extra fine‑tuning pass to recover accuracy loss. Empirically, O1 + HAP provides the most efficient trade‑off.

4. Hardware efficiency – On the Jetson Orin Nano, OnDA‑1 (HAP‑before‑training) improves training latency by 1.52× and inference latency by 1.57× on the GPU, with corresponding energy reductions of 1.64× and 1.77×. On the CPU, training latency improves by 1.86× and inference latency by 1.93×, with energy savings of 1.94× and 2.07× respectively. OnDA‑2 (L1‑after‑training) also yields gains but is consistently outperformed by the data‑aware variant.

The authors conclude that coupling weight adaptation with online structured channel pruning is a viable strategy for personalized KWS on resource‑constrained devices. Data‑aware pruning, even with limited pseudo‑labeled data, can dramatically shrink model size without sacrificing detection performance, and the resulting smaller networks translate into measurable latency and power savings on real edge hardware.

Future work is suggested in three directions: (i) combining channel pruning with quantization or neural architecture search for even higher compression, (ii) developing lighter second‑order importance estimators suitable for ultra‑low‑power microcontrollers, and (iii) addressing privacy concerns associated with storing pseudo‑labeled audio on‑device. Overall, the paper provides a solid empirical foundation for on‑device architectural adaptation and opens a promising avenue for efficient, user‑tailored speech interfaces.