Semantic-Deviation-Anchored Multi-Branch Fusion for Unsupervised Anomaly Detection and Localization in Unstructured Conveyor-Belt Coal Scenes

Reliable foreign-object anomaly detection and pixel-level localization in conveyor-belt coal scenes are essential for safe and intelligent mining operations. This task is particularly challenging due to the highly unstructured environment: coal and gangue are randomly piled, backgrounds are complex and variable, and foreign objects often exhibit low contrast, deformation, occlusion, resulting in coupling with their surroundings. These characteristics weaken the stability and regularity assumptions that many anomaly detection methods rely on in structured industrial settings, leading to notable performance degradation. To support evaluation and comparison in this setting, we construct \textbf{CoalAD}, a benchmark for unsupervised foreign-object anomaly detection with pixel-level localization in coal-stream scenes. We further propose a complementary-cue collaborative perception framework that extracts and fuses complementary anomaly evidence from three perspectives: object-level semantic composition modeling, semantic-attribution-based global deviation analysis, and fine-grained texture matching. The fused outputs provide robust image-level anomaly scoring and accurate pixel-level localization. Experiments on CoalAD demonstrate that our method outperforms widely used baselines across the evaluated image-level and pixel-level metrics, and ablation studies validate the contribution of each component. The code is available at https://github.com/xjpp2016/USAD.

💡 Research Summary

**
The paper addresses the challenging problem of detecting and localizing foreign objects on conveyor‑belt coal streams, a setting that is highly unstructured and therefore unsuitable for most existing industrial anomaly‑detection methods. In such scenes, coal and gangue are randomly piled, the belt surface exhibits complex wear, dust, and illumination variations, and foreign objects (e.g., wood, metal, plastic bags) often appear with low contrast, deformation, and partial occlusion. These factors break the assumptions of stable appearance, regular layout, and limited background variation that underlie many memory‑based, reconstruction‑based, or teacher‑student approaches, leading to severe performance drops.

To enable systematic research, the authors construct a new benchmark called CoalAD. It contains 2,490 normal training images and 1,754 test images (943 of which contain anomalies), together with pixel‑level ground‑truth masks. The dataset captures a wide range of real‑world disturbances such as dust, belt wear, and lighting changes, providing a realistic testbed for unsupervised anomaly detection in unstructured environments.

The core contribution is a three‑branch complementary‑cue fusion framework that extracts anomaly evidence from three distinct perspectives:

1. Object‑level branch – Using DINOv2, a pre‑trained Vision Transformer, patch tokens from normal samples are clustered into foreground (coal and gangue) and background (conveyor belt) distributions. During inference, tokens that do not belong to either distribution are flagged as anomalous object regions. This high‑level composition modeling is robust to low‑contrast and occluded objects because it relies on global object semantics rather than local texture alone.

2. Semantic‑level branch – The global CLS token of DINOv2 is modeled with a multivariate Gaussian on normal data, providing an image‑level anomaly score (global semantic deviation). To obtain spatial localization, the authors devise a closed‑form “ablation‑based contribution analysis” that quantifies how much each patch contributes to the overall semantic deviation. The resulting contribution map directly highlights regions responsible for the semantic anomaly, effectively translating a global logical inconsistency into a pixel‑wise signal.

3. Texture‑level branch – A ResNet backbone extracts fine‑grained features, and a PatchCore‑style nearest‑neighbor matching against a memory bank of normal patches yields a texture‑level anomaly map. This branch captures subtle texture or surface defects that may be missed by the higher‑level branches.

The three anomaly maps are fused through probabilistic weighting and smoothing to produce a final localization map. For image‑level detection, the framework combines (i) the global semantic deviation distance, (ii) a spatial cue aggregated from the fused localization map, and (iii) the texture‑level anomaly score, resulting in a multi‑evidence decision that is both robust and interpretable.

Extensive experiments on CoalAD compare the proposed method against state‑of‑the‑art baselines from the three major families of unsupervised anomaly detection: memory‑based (PatchCore, RD4AD/RD++), reconstruction‑based (DRAEM), and teacher‑student distillation (EfficientAD, SimpleNet). The proposed approach outperforms all baselines across image‑level AUROC, AUPRO, and pixel‑level IoU/AUROC, achieving improvements of 4–12 percentage points. Ablation studies confirm that each branch contributes meaningfully: the semantic‑level contribution analysis improves image‑level AUROC by 2.3 % and pixel‑level IoU by 3.1 %; the texture branch adds over 5 % gain for fine‑grained defects; and the object‑level token clustering is especially effective for low‑contrast, partially occluded objects.

The system leverages pre‑trained DINOv2 and ResNet models without additional fine‑tuning, keeping training overhead low. Inference runs at approximately 30 fps on a single GPU for 1080p images, demonstrating feasibility for real‑time deployment in mining operations.

In conclusion, the paper makes three major contributions: (1) the CoalAD dataset, filling a gap for unstructured industrial anomaly detection; (2) a multi‑cue fusion architecture that unifies object composition, semantic attribution, and texture evidence; and (3) a demonstration that anchoring image‑level detection on global semantic deviation while enriching it with localization‑aware cues yields superior performance in highly challenging scenes. The authors suggest future work on lighter transformer backbones, temporal modeling across video frames, and multimodal sensor fusion (e.g., thermal or acoustic) to further enhance robustness in mining environments.