Allure of Craquelure: A Variational-Generative Approach to Crack Detection in Paintings

Recent advances in imaging technologies, deep learning and numerical performance have enabled non-invasive detailed analysis of artworks, supporting their documentation and conservation. In particular, automated detection of craquelure in digitized paintings is crucial for assessing degradation and guiding restoration, yet remains challenging due to the possibly complex scenery and the visual similarity between cracks and crack-like artistic features such as brush strokes or hair. We propose a hybrid approach that models crack detection as an inverse problem, decomposing an observed image into a crack-free painting and a crack component. A deep generative model is employed as powerful prior for the underlying artwork, while crack structures are captured using a Mumford–Shah-type variational functional together with a crack prior. Joint optimization yields a pixel-level map of crack localizations in the painting.

💡 Research Summary

The paper introduces a novel hybrid method for detecting craquelure (cracks) in digitized paintings by formulating the task as an inverse problem. The observed RGB image U is modeled as the superposition of a crack‑free background B and a crack component C (or binary mask v), possibly corrupted by noise. Because this decomposition is ill‑posed, the authors impose two Bayesian priors: a painting prior that constrains B to lie on a learned manifold of realistic artworks, and a crack prior that encourages v to capture thin, elongated structures typical of cracks.

The painting prior is realized with a deep generative model, specifically a VQ‑GAN. After pre‑training on ImageNet, the VQ‑GAN is fine‑tuned on a curated set of roughly 8,100 crack‑free painting patches, adjusting only the last two encoder blocks. This yields a generator G(z) that maps a low‑dimensional latent vector z to a high‑quality, texture‑preserving reconstruction of a plausible crack‑free painting. By restricting B to the range of G, the solution space is dramatically reduced, mitigating the ambiguity inherent in the inverse problem.

For the crack prior, the authors adopt the Mumford‑Shah functional, a classic variational model for piecewise‑smooth images with discontinuities. To make it computationally tractable, they employ the Ambrosio‑Tortorelli (AT) relaxation, introducing a continuous edge indicator v(x)∈