Variability in Performance of a Machine-Learning Seismicity Catalog: Central Italy, 2016-2017

Machine learning (ML) catalogs contain many more earthquakes than routine catalogs, but their performance in phase picking and earthquake detection has not been fully evaluated. We develop station-level detection probabilities using logistic regression and combine them across a seismic network to compute spatial magnitude-of-completeness fields. We apply this approach to two catalogs from the 2016-2017 Central Italy sequence that were constructed from the same seismic network, one routine and one ML based. At the station level, the ML picker increases detection sensitivity by identifying smaller magnitude events and detecting earthquakes at greater distances. Spatially, the magnitude-of-completeness decreases substantially, with median values shifting from 1.6 to 0.5 for P waves and from 1.7 to 0.5 for S waves. However, the ML catalog also shows greater variability in station-level performance than the routine catalog. These results demonstrate that ML-based improvements in detectability are widespread but spatially non-uniform, highlighting their benefits, their limitations, and the potential for further improvements.

💡 Research Summary

This paper presents a quantitative comparison of detection performance between a conventional (routine) seismic catalog and a machine‑learning (ML) catalog generated with the PhaseNet deep‑learning picker for the 2016‑2017 Central Italy earthquake sequence. Both catalogs are built from the same set of stations (the INGV permanent network plus the temporary stations deployed immediately after the Amatrice mainshock), eliminating bias from differing network coverage. The routine catalog contains about 82 k events (M L 0.0–6.12) while the ML catalog contains roughly 900 k events (M L –2.6–6.1).

The authors develop a probability‑based magnitude‑of‑completeness (PMC) framework that first estimates station‑level detection probabilities as a smooth function of event magnitude and hypocentral distance. Instead of the traditional binning approach, they fit a logistic‑regression model for each station and each phase (P or S), yielding a continuous probability surface pϕi(M*,Li)=1/