CMOOD: Concept-based Multi-label OOD Detection

How can models effectively detect out-of-distribution (OOD) samples in complex, multi-label settings without extensive retraining? Existing OOD detection methods struggle to capture the intricate semantic relationships and label co-occurrences inherent in multi-label settings, often requiring large amounts of training data and failing to generalize to unseen label combinations. While large language models have revolutionized zero-shot OOD detection, they primarily focus on single-label scenarios, leaving a critical gap in handling real-world tasks where samples can be associated with multiple interdependent labels. To address these challenges, we introduce COOD, a novel zero-shot multi-label OOD detection framework. COOD leverages pre-trained vision-language models, enhancing them with a concept-based label expansion strategy and a new scoring function. By enriching the semantic space with both positive and negative concepts for each label, our approach models complex label dependencies, precisely differentiating OOD samples without the need for additional training. Extensive experiments demonstrate that our method significantly outperforms existing approaches, achieving approximately 95% average AUROC on both VOC and COCO datasets, while maintaining robust performance across varying numbers of labels and different types of OOD samples.

💡 Research Summary

The paper introduces CMOOD (Concept‑based Multi‑label OOD Detection), a zero‑shot framework designed to detect out‑of‑distribution (OOD) samples in multi‑label visual tasks without any additional training. Existing OOD detectors, such as Maximum Softmax Probability, Mahalanobis distance, and recent language‑model‑based methods like NegLabel and NegPrompt, are primarily built for single‑label classification and assume that OOD classes are semantically far from in‑distribution (ID) classes. These assumptions break down when multiple labels co‑occur, when novel combinations of known labels appear, or when subtle semantic shifts occur.

CMOOD tackles these challenges by leveraging a pre‑trained vision‑language model (CLIP) together with a large language model (LLM) to expand the original label set into two complementary concept sets: positive concepts (P) and negative concepts (N). Positive concepts are generated by prompting GPT‑4 to produce fine‑grained attributes, super‑classes, and commonly associated items for each base label. The resulting textual descriptors capture detailed, domain‑relevant semantics that remain closely aligned with the ID classes. Negative concepts are mined from a lexical resource (e.g., WordNet); each candidate word’s embedding is compared to the embeddings of all base labels, and those with the lowest cosine similarity are retained, ensuring maximal semantic distance from the ID space.

Both concept sets are embedded using CLIP’s text encoder, while an input image is encoded with CLIP’s image encoder to obtain a visual embedding h. For each of the three sets (B – the original labels, P, and N) CMOOD computes the top‑k mean cosine similarity between h and the text embeddings, denoted µk(B, I), µk(P, I), and µk(N, I). The final ID score is a weighted combination:

S_ID(I) = α·µk(B, I) + β·µk(P, I) – γ·µk(N, I)

where α, β, and γ are hyper‑parameters tuned on a validation split. If S_ID falls below a threshold γ_thr, the sample is classified as OOD; otherwise it is considered ID. This scoring function simultaneously rewards alignment with known concepts and penalizes similarity to deliberately distant concepts, sharpening the decision boundary even when OOD samples share many attributes with ID classes.

The authors evaluate CMOOD on two large multi‑label benchmarks, PASCAL VOC (20 classes) and MS‑COCO (80 classes), constructing several OOD scenarios: (1) completely unseen objects, (2) novel combinations of existing labels, and (3) varying numbers of active labels per image. Metrics include AUROC, AUPR, and FPR@95TPR. CMOOD achieves an average AUROC of about 95%, outperforming MSP (~85%), Mahalanobis (~88%), NegLabel (~90%), and NegPrompt (~91%). Notably, performance remains stable across the different OOD scenarios, demonstrating the method’s robustness to label co‑occurrence and subtle distribution shifts.

Ablation studies reveal that (i) both positive and negative concept sets contribute significantly—removing either degrades AUROC by 3–5 points; (ii) the top‑k value of 10 provides the best trade‑off between sensitivity and noise robustness; (iii) the method’s throughput is approximately 800 images per second on a CLIP‑B/16 backbone, confirming its suitability for real‑time applications.

Limitations discussed include dependence on the quality of LLM‑generated prompts and potential scarcity of truly distant negative concepts in highly specialized domains. Future work proposes automated prompt optimization, domain‑specific lexical resources, and human‑in‑the‑loop verification to further enhance concept quality.

In summary, CMOOD presents a novel paradigm—concept‑based label expansion combined with a contrastive top‑k scoring mechanism—that effectively bridges the gap between single‑label OOD detection research and the practical demands of multi‑label visual systems, delivering zero‑shot, high‑accuracy OOD detection without any model retraining.