Measuring the Dark Matter Self-Interaction Cross-Section with Deep Compact Clustering for Robust Machine Learning Inference

We have developed a machine learning algorithm capable of detecting out-of-domain data'' for trustworthy cosmological inference. By using data from two separate suites of cosmological simulations, we show that our algorithm is able to determine whether observed’’ data is consistent with its training domain, returning confidence estimates as well as accurate parameter estimations. We apply our algorithm to two-dimensional images of galaxy clusters from the BAHAMAS-SIDM and DARKSKIES simulations with the aim to measure the self-interaction cross-section of dark matter. Through deep compact clustering we construct an informative latent space where galaxy clusters are mapped to the latent space forming ``latent-clusters’’ for each simulation, with the location of the latent-cluster corresponding to the macroscopic parameters, such as the cross-section, $σ_{\rm DM}/m$. We then pass through mock observations, where the location of the observed latent-cluster informs us of which properties are shared with the training data. If the observed latent-cluster shares no similarities with latent-clusters from the known simulations, we can conclude that our simulations do not represent the observations and discard any parameter estimations, thus providing us with a method to measure machine learning confidence. This method serves as a blueprint for transparent and robust inference that is in demand in scientific machine learning.

💡 Research Summary

This paper presents a novel machine‑learning framework for measuring the self‑interaction cross‑section of dark matter (σ_DM/m) using images of galaxy clusters. The authors combine two large cosmological simulation suites—BAHAMAS‑SIDM and DARKSKIES—to train a model that can both estimate σ_DM/m and assess whether a given observation lies within the domain of the training data.

The input data consist of projected total‑mass maps (including dark matter, stars, gas, and black holes) binned to 100 × 100 pixels with 20 kpc resolution, covering a 2 Mpc radius around each cluster. These maps are chosen because they can be approximated from weak‑lensing observations and contain the information needed to discriminate between collisionless CDM and various SIDM models.

The core algorithm is “deep compact clustering.” First, a convolutional encoder (based on ResNet‑50 blocks) extracts high‑dimensional features from the images. These features are then compressed into a low‑dimensional latent space (≈10 dimensions) using a semi‑supervised learning scheme. In this latent space each simulation model (characterized by a specific σ_DM/m value) is treated as a distinct class, while the real Universe is represented as an additional class with an unknown σ. A contrastive‑plus‑cross‑entropy loss simultaneously pulls together samples from the same simulation and pushes apart different simulations, forming well‑separated clusters whose centroids correspond to the true σ values.

During inference, a mock observation is passed through the encoder, mapped to the latent space, and its distance to each cluster centroid is measured. The nearest centroid provides a regression estimate of log10 σ_DM/m, while the Euclidean distance to the nearest centroid is transformed into a confidence score. If the observation lies farther than a predefined threshold (e.g., three standard deviations) from all trained centroids, the model flags it as out‑of‑domain and discards the σ estimate. This mechanism directly addresses the “confidence‑wrong” problem that plagues conventional neural networks, which otherwise output a value even for completely unseen data.

To handle the fact that CDM has σ = 0, the authors assign an effective cross‑section of 0.01 cm² g⁻¹ to all CDM simulations, ensuring a smooth transition between the zero‑cross‑section and low‑σ SIDM regimes. The regression target is the logarithm of σ, which improves learning across several orders of magnitude.

The authors evaluate the method in three ways. First, they train on BAHAMAS‑SIDM and test on unseen DARKSKIES simulations (σ = 0.1 cm² g⁻¹). The model recovers the true cross‑section with a mean absolute error of ≈0.12 cm² g⁻¹ and correctly reports high confidence. Second, they introduce a synthetic SIDM model with σ = 0.5 cm² g⁻¹ that was not present in the training set; the confidence score drops below the threshold, and the estimate is automatically ignored, demonstrating robust out‑of‑domain detection. Third, they compare against the earlier classification‑only approach (Harvey 2024) that used an Inception‑v4 network to classify three discrete models; their regression‑plus‑clustering pipeline yields continuous σ estimates with quantified uncertainties and superior generalisation across simulation suites.

Technical details include training with Adam (learning rate 1e‑4), batch size 64, and Bayesian hyper‑parameter optimisation (Optuna) for latent dimension, number of clusters, and loss weighting. The loss function combines a contrastive term (encouraging intra‑class compactness) with a cross‑entropy term (preserving known class labels). The code, trained weights, and data preprocessing scripts are released publicly (https://github.com/EthanTreg/Bayesian‑DARKSKIES), facilitating reproducibility.

In conclusion, the paper makes four key contributions: (1) a semi‑supervised deep clustering architecture that interpolates between sparsely sampled σ values, (2) an explicit confidence metric that flags out‑of‑domain observations, (3) a regression framework that yields continuous SIDM cross‑section estimates rather than discrete class probabilities, and (4) a practical blueprint for applying trustworthy machine learning to upcoming large‑scale cluster surveys (e.g., Euclid, Rubin). Future work could extend the method to velocity‑dependent SIDM models, incorporate strong‑lensing constraints, and test on real observational data.