Audio Deepfake Detection at the First Greeting: "Hi!"

This paper focuses on audio deepfake detection under real-world communication degradations, with an emphasis on ultra-short inputs (0.5-2.0s), targeting the capability to detect synthetic speech at a conversation opening, e.g., when a scammer says “Hi.” We propose Short-MGAA (S-MGAA), a novel lightweight extension of Multi-Granularity Adaptive Time-Frequency Attention, designed to enhance discriminative representation learning for short, degraded inputs subjected to communication processing and perturbations. The S-MGAA integrates two tailored modules: a Pixel-Channel Enhanced Module (PCEM) that amplifies fine-grained time-frequency saliency, and a Frequency Compensation Enhanced Module (FCEM) to supplement limited temporal evidence via multi-scale frequency modeling and adaptive frequency-temporal interaction. Extensive experiments demonstrate that S-MGAA consistently surpasses nine state-of-the-art baselines while achieving strong robustness to degradations and favorable efficiency-accuracy trade-offs, including low RTF, competitive GFLOPs, compact parameters, and reduced training cost, highlighting its strong potential for real-time deployment in communication systems and edge devices.

💡 Research Summary

The paper tackles a practical yet under‑explored problem in audio deepfake detection (ADD): identifying synthetic speech within the first second of a conversation when the signal has already been degraded by real‑world communication processes such as codec compression and packet loss. While most recent ADD research focuses on relatively long (3–4 s) clean recordings, the authors argue that real‑time security systems need to react instantly—ideally as soon as the interlocutor says “Hi”. To this end they introduce Short‑MGAA (S‑MGAA), a lightweight extension of the Multi‑Granularity Adaptive Time‑Frequency Attention (MGAA) architecture, specifically engineered for ultra‑short (0.5–2.0 s) utterances under realistic degradations.

The core contribution lies in two novel modules that are inserted before and after the original MGAA block:

1. Pixel‑Channel Enhanced Module (PCEM) – This component jointly models fine‑grained pixel‑level saliency, channel‑wise importance, and the interaction between frequency and time dimensions. It consists of a Pixel‑Level Detector (a 3×3 depthwise convolution followed by batch‑norm, GELU, and sigmoid), a Channel‑Wise Amplifier (global average pooling → 1×1 bottleneck → expansion → sigmoid), and a Time‑Frequency Coupling block (factorized 1×3 and 3×1 convolutions). The three outputs are multiplied element‑wise and passed through a pointwise convolution to produce an enhanced feature map that highlights subtle forgery cues even when the overall signal is heavily compressed.

2. Frequency Compensation Enhanced Module (FCEM) – Because ultra‑short clips contain insufficient temporal dynamics, FCEM compensates by extracting richer information from the frequency axis. It builds three parallel frequency‑scale branches (1‑D convolutions with kernel sizes 20, 15, and 10) and three adaptive pooling paths (two max‑pool, one average‑pool). After resizing, the branches are concatenated, fused with a 1×1 convolution, and finally modulated by a frequency‑time attention map generated via a 7×1 depthwise convolution. This design injects multi‑scale spectral context, effectively “stretching” the limited time axis.

The overall pipeline processes three common time‑frequency representations—LFCC, CQCC, and MFCC—through the sequence: PCEM → MGAA block → FCEM → convolutional feature embedding blocks (CFEB‑64, CFEB‑128) → a second S‑MGAA stage → flatten → binary classifier. The authors train the system on a massive composite dataset (Dcom) comprising 640 k genuine and 1.19 M synthetic utterances drawn from six public corpora, augmented with 30 types of communication degradations (different codecs, five packet‑loss rates). Evaluation uses the ADD‑C test set, which contains six conditions (C0 clean to C5 severe degradation) and reports Equal Error Rate (EER) as the primary metric.

Key experimental findings:

• Performance across durations: For the hardest 0.5 s condition, S‑MGAA‑MFCC achieves an average EER of 2.70 %, a 23.9 % absolute reduction over the strongest baseline (RawGAT‑ST at 5.60 %). Similar gains are observed for 1 s, 1.5 s, and 2 s inputs, confirming that the proposed modules consistently improve detection even as more temporal information becomes available.
• Feature‑agnostic gains: Compared with the original MGAA, S‑MGAA reduces EER by 28–71 % across all three feature types, demonstrating that PCEM and FCEM are not tied to a specific front‑end.
• Robustness to degradations: The average EER across all six communication conditions (C0–C5) remains low, indicating that the attention‑based saliency amplification and frequency compensation effectively mitigate codec‑induced spectral smoothing and packet‑loss artifacts.
• Efficiency: S‑MGAA requires only 0.02–0.08 GFLOPs, 0.99–2.14 M parameters, and 0.25–0.49 h of training time for the four duration settings. Real‑Time Factor (RTF) stays between 0.38 and 0.10, far more stable than the original MGAA (RTF 1.75–4.82 at 0.5 s). This lightweight footprint makes the model suitable for deployment on smartphones, edge routers, or embedded voice assistants.
• Ablation study: Removing PCEM or FCEM degrades performance by roughly 5–10 % absolute EER, confirming each module’s contribution. Placing S‑MGAA only in shallow or deep layers also harms accuracy, suggesting that distributing the enhancement across the network yields the best trade‑off between low‑level detail and high‑level abstraction.

Implications and future directions:
The work demonstrates that ultra‑short, real‑time deepfake detection is feasible when attention mechanisms are carefully engineered to amplify fine‑grained spectral cues and to compensate for missing temporal context. The modular nature of PCEM and FCEM means they could be grafted onto other time‑frequency models (e.g., ResNet‑based spectrogram classifiers) or extended to multimodal settings (audio‑visual deepfake detection). Moreover, the authors’ comprehensive degradation pipeline provides a realistic benchmark for future research, encouraging the community to move beyond clean, long‑duration datasets.

In summary, S‑MGAA offers a compelling combination of accuracy, robustness, and computational efficiency for detecting synthetic speech at the very start of a conversation, paving the way for proactive anti‑spoofing defenses in real‑world communication infrastructures.