Atomic Depth Estimation From Noisy Electron Microscopy Data Via Deep Learning

We present a novel approach for extracting 3D atomic-level information from transmission electron microscopy (TEM) images affected by significant noise. The approach is based on formulating depth estimation as a semantic segmentation problem. We address the resulting segmentation problem by training a deep convolutional neural network to generate pixel-wise depth segmentation maps using simulated data corrupted by synthetic noise. The proposed method was applied to estimate the depth of atomic columns in CeO2 nanoparticles from simulated images and real-world TEM data. Our experiments show that the resulting depth estimates are accurate, calibrated and robust to noise.

💡 Research Summary

The paper introduces a novel deep‑learning framework, named SegDepth, for estimating the depth of atomic columns in transmission electron microscopy (TEM) images that are heavily corrupted by Poisson noise. Traditional electron tomography requires a tilt series of many images and long acquisition times, which makes it unsuitable for studying dynamic processes at the atomic scale. The authors therefore recast the depth‑estimation problem as a semantic‑segmentation task: each pixel is assigned an integer label (0–10) corresponding to the number of atoms in the column that projects onto that pixel, with 0 representing background.

To train the network, a large synthetic dataset is generated. Atomic models of CeO₂ nanoparticles in the (110) zone‑axis orientation are built with varying surface morphologies (flat, saw‑tooth, stepped) and thicknesses ranging from 3 to 10 atoms. Multi‑slice TEM simulations are performed using Dr. Probe, sampling a realistic range of microscope parameters (defocus 1–9 nm, C₃ = –9 µm, C₅ = 5 mm, etc.). The clean simulated images are then corrupted with Poisson noise; the noise level is controlled by a parameter λ, allowing the authors to emulate both high‑SNR and low‑SNR experimental conditions. For each simulated image, a depth mask is created by projecting all atoms onto the imaging plane and counting how many fall onto each pixel.

The segmentation network adopts a UNet architecture with six down‑sampling / up‑sampling stages, followed by two additional down‑sampling layers to match the output dimension to the 11‑class label space. A 4 × 4 median filter is applied to the raw logits to enforce spatial consistency, after which a soft‑max layer yields per‑pixel class probabilities. Training uses a cross‑entropy loss with class‑balance weighting to mitigate the dominance of background pixels.

Quantitative evaluation on simulated data shows a mean absolute error (MAE) below 0.2 atoms, pixel‑wise accuracy above 93 %, and well‑calibrated probability estimates (calibration error < 5 %). The model’s robustness to noise is demonstrated by testing across a wide range of λ values; performance degrades only minimally as SNR decreases. When applied to real TEM video frames of CeO₂ nanoparticles, SegDepth outperforms a conventional image‑simulation matching approach by roughly 30 % in depth‑prediction accuracy. Confidence scores, derived from the entropy of the soft‑max distribution, are high across most of the particle interior but drop near edges and thin regions, providing a useful indicator of prediction uncertainty.

The authors discuss several limitations. The current label set (0–10 atoms) restricts applicability to thicker specimens; extending the range would exacerbate class imbalance. The study does not quantitatively assess the impact of mismatches between simulated and actual microscope parameters beyond the explored ranges. Finally, the method processes each frame independently, so temporal coherence in dynamic experiments is not exploited. Future work is suggested to incorporate recurrent or transformer‑based architectures (e.g., ConvLSTM, Vision Transformers) to capture time‑series information and to broaden the label space for thicker samples.

In summary, SegDepth demonstrates that a carefully designed semantic‑segmentation network, trained on physics‑based simulated data with realistic noise, can reliably infer atomic‑column depth from single noisy TEM images. This capability opens the door to real‑time, three‑dimensional characterization of dynamic nanomaterials, marking a significant step forward for deep‑learning applications in electron microscopy.