Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection

Modern machine learning models deployed in the wild can encounter both covariate and semantic shifts, giving rise to the problems of out-of-distribution (OOD) generalization and OOD detection respectively. While both problems have received significant research attention lately, they have been pursued independently. This may not be surprising, since the two tasks have seemingly conflicting goals. This paper provides a new unified approach that is capable of simultaneously generalizing to covariate shifts while robustly detecting semantic shifts. We propose a margin-based learning framework that exploits freely available unlabeled data in the wild that captures the environmental test-time OOD distributions under both covariate and semantic shifts. We show both empirically and theoretically that the proposed margin constraint is the key to achieving both OOD generalization and detection. Extensive experiments show the superiority of our framework, outperforming competitive baselines that specialize in either OOD generalization or OOD detection. Code is publicly available at https://github.com/deeplearning-wisc/scone.

💡 Research Summary

The paper addresses a practical problem that arises when machine‑learning models are deployed in uncontrolled environments: they must both generalize to covariate shifts (where the input distribution changes but the label space stays the same) and detect semantic shifts (where completely new classes appear). Existing research treats these two tasks separately, leading to methods that excel at one but fail at the other. The authors propose a unified solution called SCONE (Semantic and Covariate Out‑of‑distribution Learning via Energy Margins) that simultaneously tackles both challenges by exploiting freely available, unlabeled “wild” data collected during deployment.

Key idea and problem formulation
Wild data is modeled as a mixture of three distributions: in‑distribution (ID) data, covariate‑shifted OOD data, and semantic‑shifted OOD data. Formally, (P_{\text{wild}} = (1-\pi_c-\pi_s)P_{\text{in}} + \pi_c P_{\text{cov}} + \pi_s P_{\text{sem}}), where (\pi_c) and (\pi_s) are the proportions of covariate and semantic OOD respectively. This mixture reflects real‑world streams where the model sees a blend of familiar and unfamiliar inputs without any labels.

Why previous methods fail
The closest prior work, WOODS, also uses wild data but treats all non‑ID samples as semantic OOD. Its objective pushes the energy‑based detector to label as many wild samples as OUT as possible. Because covariate‑shifted samples share the same label space as ID data, this forces the decision boundary to cut through the region where covariate‑shifted points lie, causing them to be mis‑classified and dramatically reducing OOD generalization performance.

SCONE’s margin‑based formulation
SCONE introduces a negative energy margin (\eta < 0) for ID samples. The learning problem becomes: \