Are Modern Speech Enhancement Systems Vulnerable to Adversarial Attacks?

Machine learning approaches for speech enhancement are becoming increasingly expressive, enabling ever more powerful modifications of input signals. In this paper, we demonstrate that this expressiveness introduces a vulnerability: advanced speech enhancement models can be susceptible to adversarial attacks. Specifically, we show that adversarial noise, carefully crafted and psychoacoustically masked by the original input, can be injected such that the enhanced speech output conveys an entirely different semantic meaning. We experimentally verify that contemporary predictive speech enhancement models can indeed be manipulated in this way. Furthermore, we highlight that diffusion models with stochastic samplers exhibit inherent robustness to such adversarial attacks by design.

💡 Research Summary

The paper investigates whether modern speech‑enhancement (SE) systems are vulnerable to targeted adversarial attacks. While traditional Wiener‑filter based enhancers lack the expressive power to alter semantic content, recent deep learning approaches—both predictive (direct mapping and complex‑ratio masking) and generative (score‑based diffusion)—are sufficiently expressive to be exploited. Assuming a white‑box threat model where the attacker knows the full architecture and parameters, the authors formulate an attack that adds a complex‑valued perturbation δ to a noisy mixture Y_user so that the enhanced output f_SE(Y_user + δ) closely matches a pre‑chosen target utterance S_attacker rather than the original speech S_user.

The core loss is a simple mean‑squared error between the enhanced output and the target speech. To keep the perturbation imperceptible, the authors incorporate a psychoacoustic masking model based on the MPEG‑1 standard. For each time‑frequency bin they compute an audibility threshold H(q,n) and the spectral magnitude of the perturbation D(q,n). The difference Φ = H − D (plus a tolerance λ) yields a mask ˆΦ∈