CVA: Context-aware Video-text Alignment for Video Temporal Grounding

We propose Context-aware Video-text Alignment (CVA), a novel framework to address a significant challenge in video temporal grounding: achieving temporally sensitive video-text alignment that remains robust to irrelevant background context. Our framework is built on three key components. First, we propose Query-aware Context Diversification (QCD), a new data augmentation strategy that ensures only semantically unrelated content is mixed in. It builds a video-text similarity-based pool of replacement clips to simulate diverse contexts while preventing the ``false negative" caused by query-agnostic mixing. Second, we introduce the Context-invariant Boundary Discrimination (CBD) loss, a contrastive loss that enforces semantic consistency at challenging temporal boundaries, making their representations robust to contextual shifts and hard negatives. Third, we introduce the Context-enhanced Transformer Encoder (CTE), a hierarchical architecture that combines windowed self-attention and bidirectional cross-attention with learnable queries to capture multi-scale temporal context. Through the synergy of these data-centric and architectural enhancements, CVA achieves state-of-the-art performance on major VTG benchmarks, including QVHighlights and Charades-STA. Notably, our method achieves a significant improvement of approximately 5 points in Recall@1 (R1) scores over state-of-the-art methods, highlighting its effectiveness in mitigating false negatives.

💡 Research Summary

The paper introduces Context‑aware Video‑text Alignment (CVA), a novel framework designed to overcome a persistent problem in video temporal grounding (VTG): models often over‑associate textual queries with static background scenes, leading to poor temporal precision. CVA tackles this issue through three tightly integrated components: (1) Query‑aware Context Diversification (QCD), a data‑augmentation strategy that replaces background clips with semantically unrelated ones while explicitly avoiding false‑negative samples; (2) Context‑invariant Boundary Discrimination (CBD) loss, a contrastive objective that forces the representations at the start and end boundaries of a target moment to remain consistent despite diverse contextual perturbations; and (3) Context‑enhanced Transformer Encoder (CTE), a hierarchical transformer that combines windowed self‑attention for local temporal patterns with bidirectional cross‑attention to fuse global query information.

Query‑aware Context Diversification (QCD)
Traditional context‑mixing methods randomly sample replacement clips, which can inadvertently select clips that are semantically related to the query, turning them into misleading negatives. CVA first computes video‑clip and text‑query embeddings for the entire dataset using a pre‑trained CLIP model. Cosine similarities between every clip and every query are collected, and two distributions are formed: one for ground‑truth (GT) pairs and one for non‑GT pairs. Rather than using fixed similarity thresholds, CVA adopts percentile‑based bounds: the lower bound (θ_min) is the α‑percentile of the non‑GT distribution, filtering out clips that are too dissimilar to provide learning signal; the upper bound (θ_max) is the β‑percentile of the GT distribution, discarding clips that are too similar and could become false negatives. For each training instance, a candidate pool of replacement clips is built from a randomly chosen video B, retaining only those whose similarity to the current query falls within