Designing lensless imaging systems to maximize information capture

Mask-based lensless imaging uses an optical encoder (e.g. a phase or amplitude mask) to capture measurements, then a computational decoding algorithm to reconstruct images. In this work, we evaluate and design lensless encoders based on the information content of their measurements using mutual information estimation. Our approach formalizes the object-dependent nature of lensless imaging and quantifies the interdependence between object sparsity, encoder multiplexing, and noise. Our analysis reveals that optimal encoder designs should tailor encoder multiplexing to object sparsity for maximum information capture, and that all optimally-encoded measurements share the same level of sparsity. Using mutual information-based optimization, we design information-optimal encoders for compressive imaging of fixed object distributions. Our designs demonstrate improved downstream reconstruction performance for objects in the distribution, without requiring joint optimization with a specific reconstruction algorithm. We validate our approach experimentally by evaluating lensless imaging systems directly from captured measurements, without the need for image formation models, reconstruction algorithms, or ground truth data. Our comprehensive analysis establishes design and engineering principles for lensless imaging systems, and offers a model for the study of general multiplexing systems, especially those with object-dependent performance.

💡 Research Summary

This paper introduces a principled, decoder‑independent framework for evaluating and designing mask‑based lensless imaging systems by directly estimating the mutual information (MI) between the captured noisy measurements and the underlying objects. Traditional optical design focuses on mapping each object point to a sensor point, but lensless imagers deliberately employ multiplexed measurements through a thin phase or amplitude mask, enabling compact form factors and compressive sensing capabilities. However, existing encoder designs rely on heuristics (e.g., Gaussian diffusers, lenslet arrays) or end‑to‑end optimization that couples the encoder to a specific reconstruction algorithm, making the designs fragile to changes in decoders and computationally intensive to obtain.

The authors model the imaging pipeline as a Markov chain O → X → Y, where O denotes the object distribution, X the noiseless encoded image obtained by convolving the object with the point‑spread function (PSF) defined by the mask, and Y the noisy sensor measurement after photon detection. By invoking the Data Processing Inequality and noting that the encoding step is deterministic, they prove that I(O;Y) = I(X;Y), allowing the mutual information to be estimated solely from the measurement distribution without requiring ground‑truth objects.

MI is decomposed as I(X;Y) = H(Y) – H(Y|X). The entropy H(Y) of the noisy measurements is approximated by fitting a powerful autoregressive model (PixelCNN) to a large set of measurement patches (10 k training, 1.5 k test) and computing the cross‑entropy on a held‑out set, providing an upper bound on H(Y). The conditional entropy H(Y|X) captures the randomness introduced by detection noise; assuming independent Poisson (shot) noise that can be approximated as Gaussian at high photon counts, a closed‑form expression involving the per‑pixel mean intensity is used. The final MI estimate is the difference between these two terms, with 95 % confidence intervals obtained via bootstrap.

To explore the interaction between object sparsity and encoder multiplexing, the authors quantify sparsity using the Tamura coefficient (TC) applied to image gradients, a scalar metric that rises with increasing sparsity. Encoder multiplexing is represented by the number of non‑zero points in the PSF, realized experimentally as the number of lenslets (1–9) in a lenslet‑array mask, each lenslet modeled as a Gaussian focal spot. By sweeping object sparsity (controlled synthetic datasets) and multiplexing level, they observe a clear trade‑off: higher multiplexing yields maximal MI at higher object sparsity, whereas dense objects achieve higher MI with fewer lenslets. Remarkably, all optimal configurations converge to the same measurement sparsity (identical TC), suggesting that an information‑optimal encoder balances the sparsity of the captured measurements regardless of the underlying object distribution.

Leveraging these insights, the paper formulates an MI‑based optimization problem to design phase masks tailored to specific object distributions (sparse, moderate, dense). The optimized masks are fabricated and tested on a real lensless prototype. Reconstruction performance is evaluated with both classic total‑variation (TV) regularization and modern deep‑learning decoders. Across all decoders, the MI‑optimized masks consistently outperform heuristic masks, delivering average PSNR gains of ~2.3 dB and SSIM improvements of ~0.07, with the most pronounced benefits for sparse scenes. Crucially, the evaluation and design process does not require any ground‑truth images or knowledge of the reconstruction algorithm, making the approach robust to future decoder advances.

The authors also discuss extensions of the framework to other multiplexing systems such as coded aperture imaging, compressive spectral cameras, and multi‑modal sensors, where object‑dependent performance is similarly critical. By providing a quantitative bound on the fundamental information that can survive the encoding and detection stages, mutual information serves as a universal metric for guiding hardware design, establishing performance limits, and informing practical engineering trade‑offs.

In summary, the paper (1) introduces mutual information as a decoder‑independent metric for lensless encoder evaluation, (2) reveals a sparsity‑multiplexing trade‑off and a universal measurement sparsity at optimality, (3) demonstrates data‑driven design of phase masks that achieve superior reconstruction without joint optimization with decoders, and (4) validates the approach experimentally, thereby offering a solid theoretical and practical foundation for next‑generation lensless imaging systems.