Noninvasive super-resolution imaging through scattering media

1 N o n - i n v a s i v e s u p e r - r e s o l u t i o n i m a g i n g t h r o u g h d y n a m i c s c at t e r i n g m e d i a Dong Wang 1,2,# , Sujit K. Sahoo 1 ,3 ,# and Cuong Dang 1 ,* 1 Centre for Optoelec tron ics and Biophotonics (COEB), School of Electrical and Electronic Engineering, The Photonics Institute (TPI), Nanyang Technological University Singapore, 50 Nanyang Avenue, 639798, Singapore 2 Key Laboratory of Advanced Transducers and Intelligent Control S ystem, Ministry of Education, and Shanxi Province, Col lege of Ph y sics and Optoelectronics, Taiyuan University of Technology , Taiyuan 030024, China 3 Sc hool of Electrical Sc iences, Indian I nstitute of Technolog y Goa, Goa 403401, I ndia # Dong Wang and Sujit K. Sahoo contribute equally. *Corresponding author, E-mail: HCDang@ntu.edu.sg Keywords: super-resolution imag ing, scattering m edia, biological tissue, de correlation, stochastic, localization Abstract Super-resolution imaging with advanced optical systems has been revolutionizing technical analysis in various fields from biological to physical sciences. However, many objects are hidd en by strongly scattering media such as rough wall corners or biological tissues that scram ble ligh t paths, create speckle 2 patterns and hinder object’s visualization, let alone super -resolution imaging . Here, w e realize a method to do non -invasive su per-resolution imaging through scattering media based on stochastic optical scattering localization imaging (SOSLI) technique. Simply by capturing multiple sp eckle patterns of ph oto- switchable emitters in our demonstration, the stochastic ap proach utiliz es the speckle correlation properties of scattering media to retrieve an i mage with more than five-fold resolution enhancement compared to the diffraction limit, while posing no fundam ental limit in achieving h igher spatial resolution. More importantly, we demonstrate our SOSLI to do non -invasive super-resolution imaging through not only optical diffusers, i.e. static scattering media, bu t also biological tissues, i.e. dynamic scattering me di a with decorr elation of up to 80% . Our approac h paves the w ay to non -invasively visualize various samples behind scattering media at unprecedented leve ls of detail. Introduction Optical imaging be yond the diffraction-limit resolution has enabled incredible tools to advance science and technolog y from non-inv asive investi gation of the interior of biological cells 1,2 to chemical reactions at single molecule level s 3 . S uper-resolution stimulated emission dep letion (STED) microscopy 4 has be en pro gressed rapidly to achieve three-dimensional (3D) imaging with super-high spatiotemporal precision 5,6 . For single- molecul e detection and localization approaches 7,8 , such as stochastic optical reconstruction mi croscopy (STORM) or photo -activated localization microscop y 3 (PALM), positions of photo-switchable probes are determined as centers of diffraction- limited spots. Repeating multiple imaging processes, each with a stochasticall y different subset of active fluorophores, allows positioning a number of e mitters at few- nanometer resolution so that a high-resolution image is reconstructed 9 . After thes e pioneering techniques, the field of super-resolutio n mic roscopy has deve lo ped rapidly with various other techniques 10 - 12 to bring the optica l microscop y within the resolution of electron microscopy. However, the requirement of sample transparency makes the super-resolution microsc op y techniques impossible to access objec ts, which a re hidden by stron gly scattering media (F i g. 1a and supplementary Fig. S1, S2 ) such as biological tissues, frosted glass or around rough wall corners. These m edia do not absorb light significantly ; however, they create noise-like spec k le pa tterns 13 a nd challeng e eve n our low-resolution visualization of samples. Many app roaches have been demonstrated to ov ercome the scattering ef fects and enabled imaging or focusing capability through scattering media . The most straightforward app roaches utilize ballistic photons 14 , such as optica l coherence tomography 15 or multi-photon microscop y 16 . However, strongl y scattering media significantly reduce num ber of ballistic photons and lower the signal tremendousl y 17 . Some techniques require a guide star or an access on the other side of sca tt ering media to characterize or reverse their scattering e ffects before ima ging such as wave -front shaping techniques 18 - 21 , transfer matrix measurement 22,23 . The memor y eff ect of li ght through scattering media 24,25 imply ing a shift-invariant point spreading function (PSF), 4 allows imaging b y deconvolution 26 - 28 of a speckle pattern with the PSF, which is measured invasivel y. A scattering medium with a known PSF is a scattering l ens that , in turn, is a low pass filter li ke an y conventiona l lens. Deconvolution recovers and enhanc es the transmission of high frequency components (i.e. a sharper cut-off low- pass filter) therefore, resolution is slightly higher than the diff raction limit (Fig. 1b and supplementary Fig . S2 d-f). D econvolution imaging currently provide s the best- resolution images from speckle patterns. Each m easured PSF is only v alid for one scattering medium , therefore, cannot be used for d ynamic scattering media. Non- invasive imaging through scattering medi a where the image is retrieved without characterizing s cattering media is desired for real applications. Diffuse optical tomography 29,30 and time-of-flight imaging 14,29,31 are capable of seeing though scattering layers and around corners non-invasive ly ; however, the spatial resolution is much lower than the opt ical diffraction limit. Thanks to the shift-invariance speckle- type PSF of thin scatteri ng media, the 2D ima ge and even the 3D image of a sample can be revealed non-invasively from the speckle patterns by a phase retrieval algorithm 32 - 34 . The limited performance of the algorithm a nd the camera, together with presence of noise and sample’s complexit y , usuall y makes th e ima ge retrieval process fail or converge with some artifacts and slightly low er resolution compared to the diffraction limit (Fig. 1c and supplementary Fig. S2g-i). Here, we present our st ochastic optical scattering lo calization imaging (SOSLI) technique to do non-invasive super-re solution imaging through scattering media. The 5 technique onl y requires an imaging sensor to capture speckle p atterns created b y blinking emitters behind a scattering medium; no other optics or complicated alignment are needed (Fig. 1 a, Fig . 2a, and supplementary Fig. S 1). The positions of emitters in each stochastic frame are first determined at very high precision from the corresponding speckle pattern, and then a super-resolution image of the full sample is reconstructed by superposing a series of emitter position images (F i g. 1d). We demonstrate the image reconstruction with resolution beyond the diffraction limit by a factor of five as a proof of concept, while there i s no fundamental limit for S OSLI on resolution , simil ar to current super-resolution microscop y techniques. More interestingly, the localization algorithm is based on a single-shot speckle pattern with minimum correlation among adjacent patterns; therefore, we develop adaptive SOSLI to do super-resolution imaging through d ynamic sc attering media such as fresh chicken egg shell mem brane with decorrelation of up to 80 %. Our S OSLI demonstrates a d esired t echnique to see through translucent media such as biologica l tissues or frosted g lass with unprecedented clarity. Stochastic Optical Scattering Localization Imaging (SOSLI) An object  c onsists of stochastically blinking emi tters:         , where   is the i th blinking pattern (a subset of  ) and N is the total number of the blinking patterns . After light prop agating through sc attering media, each   produces a speckle pattern   , captured b y a camera (Fig. 2a). If obje ct size is within the memory effect of the scattering media, the PSF is shift -invariant and speckle-type, therefore the speckle pattern   (Fig. 2b ) of object   preserves the object’s autocorrelation (Fig. 2c). The 6 image o f   can be retrieved from its autocorrelation by an iter ative phase retrieval algorithm 33 (Fig. 2d). The limit ation in the camera’s bit depth , photon budget and performance of phase retrieval a l gorithms in presence of image acquisit ion noise degrade the diffraction-limit resolution of this non -invasive retrieval image. However, a standard local ization algorithm 35,36 is e mplo yed to find the position of emitters at very high resolution (Fig. 2e) and remove algorithm artifac ts . Similar to other localization microscopy techniques, the precision is higher for spatially sparse emitter samples where onl y one emitter is temporally active in a diffra ction - limit ed region. The sharp and clear image    presents the precise relative emitter positions of pattern   , while losing their exact positions because    is only r etrieved from autocorrelation of   through autocorrelation of   . The estimated P SF of the scattering medi um can be retrieved by deconvolution (Fig. 2f) :     󰇛      󰇜 , which is also shifted in comparison to the actual PSF beca us e of losing exact emitter positions in    . Next, a series of clean super-resolution ima ges   󰆒 with emitter posi tions is reconstructed for a corresponding series of stochas tic patterns   by de convolution of its corresponding speckl e pattern   with the estimated  and localization as presented in Fig. 2 g. A super-resolution im age of the full sample (Fig. 2h) is now reconstructed b y superposing all individual images as:  󰆒     󰆒    , which represents object  with an arbitrary position. This principle is valid provided that PSF does not change among the group of   . For comparison, we present the t ypical simulation image (Fig. 2i) retrieved from autocorrelation of a sin gle speckle pattern, in which simulation 7 parameters are similar with the exception that all the emitters are on. The simulated diffraction li mit is about 3.2 pixels. Th is current state-of-the-art technique for non- invasive imaging throug h scattering media shows a blurry image where the low spatial frequency presents the diffraction limit of the optical system to gether with some artifact from the phase r etrieval algorithm. In contrast, the image reconstructed by S OSLI is much sharper (Fig. 2h). Super-resolution imaging through a ground glass diffuser To prove our concept, we first demonstrate SOSLI for non-invasive supe r-re solution imaging through a ground glass diffuser. Microscopic objects comprising multiple stochastic blinking emitters are created by de -magnifying projector images through a microscope objective. The de-magnif ying image of each pixel in a digital micro-mirror device (DMD) is an intermittent emitter with a size of about 1 .34 µm (Sup plementary Fig. S1a) . The microscopic object is placed 10 mm behind the ground glass d iffuser, which is kept unknown in all demonstrations. The incoherent light from the object propagating through the o ptical diffuser is r ecorded by a monochromatic camera, which is 100 mm in front of the diffuser. An iris with diameter of 1 mm is placed immedia tel y after the optical diffuser to act as the aperture of the im aging s ystem. A lar ger iris siz e enhances the diffraction limit of the imaging s ystem and achieve a sharper image (supplementary Fig. S2); however, it reduces the speckle contrast that is vital for phase retrieval approach. Figure 3 shows the experimental results fo r three different imaging approaches . 8 Single-shot non-invasive imaging through s cattering media is performed in Fig. 3a, where all emitters are on. The result is recovered from the autocorrelation of a single speckle pattern b y th e phase retrieval algorithm. The image is ver y blurred, and w e cannot distinguish 2 lines with a g ap of 4 µm between them (Fig. 3b). We can calcula t e the diffraction limit of our s ystem as          , whe re  = 550 nm and NA = 0.05. Beside the diffraction limit, the performance of the phase re trieval algorithm in the presence of experimental noise also de grades the image qualit y and limits the resolution. With the DMD projector, we can measure the PSF by turning on a single pixel at the center onl y and capture its speckle pattern. Such an “invasive guiding star ” for the PS F measurement allows us to calculate the image by deconvolution and significantly e nhances the re solution (Fig. 3c). The invasive deconvolution approach is more deterministi c, robust to the noise and enhance s the high spatial frequency components of the image. This allows us to distinguish the 2 lines with a 4-µm gap between them (Fi g. 3d). However, we still cannot see the gap of 2.68 µm between 2 lines. Most s trikingly, our super -resolution imag e reconstructed non-invasively by SOSLI is remarkably clear as presented in Fig. 3 e. We can resolve very well all the smallest feature s of our sample, i.e. 2 thin lines (1.34 µm width ) with a gap of 1.34 µm in between (Fig. 3f). The smallest sample feature is smaller than the diffraction limit b y a factor o f 5. Figure 3e-f also clearly illustrat es that the capability of S OS LI is far beyond our sample’s smallest features (pixel siz e), which are currently li mited b y the projector and optics of the sample creating system. 9 It is worth to highli ght some im portant factors in our SOS LI performance . Supplementary Fig. S3 presents more detail for the reconstruction process in which the localization is important to re move all the background noise a nd artifacts in the emitter image resulting from the phase retrieval algorithm. This leads to a better estimation of PSF for a series of deconvolution calculation after that. I n addition, the localization process also allows SOS LI to tolerate more errors in deconvolution pr ocess due to imperfect PSF estimation and noise in image acquisition (Suppl ementary Fig. S 4-5). Our approach relies on st ochastic emitter patterns to reconstruct a full object; therefore, the imag e quality is improved with more stochastic patterns (supplementary Fig. S6 ). Figure 4 presents some images of more complex objects for performance comparison among the three te chniques. Similar to Fig. 3, the complex objects are best resolved with our SOSLI approach (Fig. 4a-c), while the retrieval image from autocorrelation of a single speckle pattern shows the poorest performance that also has some artifacts (Fig. 4d-f). Th e invasive ima ging app roach b y deconvo lution shows moderate performance in Fig. 4g-i. Obviousl y, our SOSLI for non-invasi ve imaging through scattering media goes far be yond the diffraction limit and surpa ss es all the current stat e-of-the-art imaging throu gh scattering media, including both invasive and non-invasive techniques. Super-resolution imaging through a biological tissue Our SOSLI demonstrations in Fig . 3 and Fig. 4 rely on a fixed PSF for r econstruction of multiple stochastic emitter patterns; and therefore, we c annot directl y use for dynamic scattering media such as biological tissues. Figure 5a shows the decorrelation 10 behaviors of PSFs for two different s cattering media. For static sc attering media such as ground glass diffusers, the P SF is a constant pattern and the correlation of 1 is achieved for an y m easured PSFs at an y time. On the other hand, dynamic scattering media such as fresh chicken eggshell membrane, the PSF is gradually changed and the correlation with the initial one dec reases with time. I n our experiment for fresh chicken eggshell membrane, the correlation reduces from 1.0 to 0.2 after 300 m easurements, with the fastest decay rate in the first 70 measurement s (correlation decreases to 0.54 ). The reconstruction by S OSLI with a single estimated PSF shows a noisy and blurr ed image due to this decorrelation of the membrane (Fig. 5b). Supplementar y Fig. S7 presents the deconvolution image s from stochastic speckle patterns with an estimated PSF from the first speckle pattern. Obviously, the assumption of a static PS F in SOSLI does not hold in this case. For very large P SF decorrelations, the localization process cannot distinguish the emitters from strong background and artifacts in the deconvolution images, leading to the result in Fig. 5b. We introduce an adaptive approach to demonstrat e our SOS LI for super-re solution imaging through dynamic scattering media . We now utilize SOSLI to localiz e and then superpose e mitters in 50 stocha stic patterns, in which the fresh c hicke n egg sh ell membrane still ca n retain its PSF correlation of more than 60%. With a tota l collection of 300 stochastic patterns, we divide this into 11 sections, each containing 50 speckle patterns in which the first 25 patterns are ov erlapped with the previous section; the other 25 patterns are then overlapped with the next section (Supple mentar y Fig. S8 ). We can 11 directly appl y SOSLI with single estimated P SFs to reconstruct 11 super-resolution images, each one representing a part of the object. However, we cannot direc tl y superpose these 11 images to reconstruct the full image because their absolute positions are lost in each SOS LI procedure. I n fact, the relative positi ons among the imag es are the relative positions a mong their respec tive retrieved PSFs, which are achieved independently (similar to the discussion on estimated PSFs from different speckle patterns in supplementary Fig. S4 , S5 ). The correlation between two retrieved PSFs is a bright spot with bac kground ; the intensity of the bright spot presents the corre l ation between two PSFs; and the position of the brig ht spot (re lative to the center) pr esents the relative position between two PSFs. Because PSF c orrelation only reduces to a bout 65% between two adjacent sections in our experiment, t he brightest spot in correlation between two estimated P SFs is very cl ear and its center is ver y easy to loca te (simi lar to emitter localization). With this procedure, the relative positions of 11 images are identified (Supplementary Fig. S9 ). W e only need to shift our individual super- resolution images to make the relative position zero before superposing them to rece ive a full image. Figure 5c presents the superposing result after alignment of their individual images. The final image with adaptive SOS LI is super-resolution, much clearer and less noise compared to SOSLI with a static PSF (Fig. 5b). For comparison, the low-resolution images obtained by the ph ase re trieval algorithm and invasive deconvolution of this object through chicken eggshell membrane are similar to Fig . 1b- c, respectively. By doing adaptive SOSLI, we re construct a super-resolution image non- invasively throu gh dynamic scattering media with effective correlation of more than 12 0.6 (shading area in Fig. 5a) while the actual correlation reduces to 0.2 during image acquisition. The procedure can continue with mo re stochastic pattern a cquisition and the membrane is ev en completely d ecorrelated but the effective correlation for adaptive SOSLI still can maintain at more than 60%. Our SOS LI with an adaptive PSF shows a very practical approach to do non -invasive sup er-resolution imaging through d ynamic scattering media. Discussion and Conclusion Our SOS LI relies on a shift-invariant speckle-t ype PSF of scattering media, which is valid if the hidden object behind the scattering media is within the memory effect region of the scattering media . Therefore, we can image a larger object with thinner scattering media. However, there is no fundamental ph ysics to limit our SOSLI ’ s resolution , similar to conventional super-resolution microscopy. In fact, we have practical challenges for SOSLI to reach high resolution. The most challeng ing requirement is the photon budget for e ach i ntermittent emitter, i.e. the number of photons per emitter per blink. For the localiz ation approach in super-resolution microscopy 7-9 where all the photons go into the diff raction limit spot s, the r esolution can be enhanced b y   whe re  is the number of captured photons. With scattering media, we n eed to capture many more photons, because the photons are now scattered everywhere, for m speckles, and we need multiple speckles to retrieve an image. The noise and bit depth of camera, as well as the sparsity of active emitters are also im portant factors tha t affect our computational approach, limiting the resolution. 13 It is important for any imag in g technology to beat the dy namics of both the object and the environment. Like an y other localization approach 7-9 , our S OSLI requires a static object during the whole imag e acquisiti on process. However, the ada p tive SOSLI can significantl y miti gate the environment d ynamics (i.e. the decorrelation of scattering media). It is wor th to note that our method for localizing emitters in a stochastic speckle pattern relies on a single-shot imag e, and then the final super-re solution imag e relies on alignment of the localized-emitter patterns. The former implies that a stochastic speckle pattern c an be ca ptured sufficientl y fast to beat th e dynamics of the scatter ing media. The latter is more impor tant for SOSLI. In extremel y d ynamic scattering media, the deconvolution might fail even when usin g the estimated PSF for immediately next speckle p attern because of fast decorrelation. We can then apply th e most adaptive SOSLI b y retrieving ev ery emitter p attern from i ts speckle autocorrelatio n b y phase retrieval al gorithm then localiz ation. The only requirement for the most adaptive approach is that the sc attering media do not decorr elate completely between two consecutive shot s. In our demonstration (supplementary Fig . S10), only 20 % correlation is sufficient to determine the relative posit ions betwee n two super-resolution images. Superposing then can be carried out after proper alignment to achieve a super- resolution image. In summar y, we have presented our sim ulation and proof-of-concept demo nstration of SOSLI for non-invasive super-resolution imaging through both static a nd dyna mi c scattering media. W e only need a camera to capture multiple images of scatt ered light 14 from stochastic emitters behind scattering media, and then our computational approach will loca liz e these emitters non-invasivel y to reconstruct a supe r -resolution image . Our experimental results sho w that SOS LI enhances r esolution by a factor of 5 compared to the diffraction limit, showing features with considerabl y more detail compared to both state-of-the-art invasive and non-invasive imaging throug h scattering media. The demonstrated resolution enhancement is currently limited b y our sample preparation while the SOSLI technique presents no fundamental limit to achieve hig h er resolution. The adaptive S OSLI allows super-resolution imaging non-invasivel y thr ough hi ghly dynamic s cattering med ia with dec orrelation of up to 80% while capturing two consecutive speckle patterns. Our S OSLI demonstration shows a promising approach for optical imaging through d ynamic turbid media, such as biological tissue , with unprecedented clarity. Method Sc ale bar: All the experim ental results of recovered images show a scale bar of 10 camera pixels that is equivalent to 65 µm in the imag ing plane. This is corresponding to 6.5 µm on th e object plane b ecause the mag nification is 10. Howev er, we do not k now the scale bar on the object plane or mag nification of the imag ing system in non- invasive approach because the distance f rom t he object to scattering media is unknown. We can only resolve the sa mple by angle resol ution. The sc ale bar o f 65 µm in the imaging plane i s equivalent to the angle of 0.65 mrad. Data processing: In all experiments, the reso lution of the raw camera images is 2560 ×2160 15 pixels. We crop them into a resolution of 2048×2048 pixels for implementations of all the mentioned techniques in this work. The final reconstructed images ar e cropped to a square window with dim ensions ra nging between 75×75 pixels and 15 0×150 pixels (depe nding on the imaged object dim ensions). Algorithm s are develo ped in Ma tlab and run on a norm al PC (I ntel Core i7, 16 GB memory). A t ypical procedure f or SOSLI wit h 300 speckle patterns tak es 2- 3 minutes. Funding The research is financiall y supported b y Sin gapore Ministry of Health’s National Medical Research Council (NMRC) : C BRG-NIG (NMRC/BNIG/2039/2015), Ministry of Education – Singapore (MOE) : MOE-AcRF Tier-1 (MOE2017 - T1 -002-142), and Nanyang Technological University (NTU). Acknowledgements Authors speciall y thank Professor S ylvain Gigan and his research group at Laboratoire Kastler B rossel, Sorbonne Université, École Normale Supér ieure – P aris Sciences et Lettres (PS L) Research Universit y, Paris, France for great discussions and providing suggestions for improvement. We would like to thank Xiangwen Zhu, Vinh Tran, Dr. Dayan Li, D r. Dongliang Tang, and Dr. Hu y Lam at NTU Sin gapore for fruitful discussions and useful feedbacks. We would like to thank Dr. Phil ip Anthon y Surman for proof reading. 16 Author contributions S.K.S. performed the numerical simulations. C.D., D.W . and S .K.S. designe d the initial experiments. D.W. performed the ex periments wi th S.K.S. ’s participation. All authors discussed, analyzed and took responsibility for the results and content of the paper. C.D. and D.W. wrote the manuscript with S.K.S.’s contributions. C.D. supervised and contributed to all aspects of research. References 1 Schermelleh, L . et al. Super-re solution microscop y dem ystified. Nature Cell Biology 21 , 72-84, (2019). 2 Neef, J. et al. Quantitative opti cal nanoph ysiology of Ca2+ signaling at inn er hair cell active zones. Nature Communications 9 , 290, (2018). 3 Pujals, S ., Feiner-Gracia, N., Delcanale, P., Voets, I. & Albertazzi, L. S uper- resolution microscopy as a powerful tool to stud y complex s y nthetic materials. N ature Reviews Chemistry 3 , 68-84, (2019). 4 Hell, S. W. & W ichmann, J . Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscop y. Opt L ett 19 , 780- 782, (1994). 5 Vicidomini, G., Bianchini, P. & Diaspro, A. ST ED super-resolved microscopy. Nature Methods 15 , 173, (2018). 6 Hell, S. W. Toward fluorescence nanos copy. Nature Biotechnology 21 , 134 7-1355, (2003). 7 Betzig, E. Proposed m ethod for molecular optic al imaging. Opt Lett 20 , 2 37-239, (1995). 8 Betzig, E. et al. Imaging Intracellular Fluorescent Proteins at Nanometer Resolution. Science 313 , 1642-1645, (2006). 9 Balzarotti, F. et al. Nanometer resolution ima ging and tr acking of fluorescent molecules with minimal photon fluxes. Science 355 , 606-612, (2017). 17 10 Turcotte, R. et al. Dynamic super-resolution struc t ured illumination imag ing in the living brain. Pro ceedings of the National Academy of Sciences 116 , 9586-9591, (2019). 11 Tenne, R. et al. Super-resolution enhancement b y quantum image scannin g microscopy. Nat Photonics 13 , 116-122, (2019). 12 Kaldewey, T. et al. Far-field nanoscopy on a semi conductor quantum dot via a rapid-adiabatic-passage-based switch. Nat Photonics 12 , 68-72, (2018). 13 Goodman, J. W. Speckle Phenomena in Optics: Theory and Applications . (Roberts & Company, 2007). 14 Kang, S. et al. Imaging deep within a scattering medium using collective accumulation of single-scattered waves. Nat Photonics 9 , 253, (2015). 15 Siddiqui, M. et al. High-speed optical coherence tomograph y b y circular interferometric ranging. Nat Photonics 12 , 111-116, (2018). 16 Hoover, E. E. & Squier, J. A. Advances in multiphoton microscop y technolog y. Nat Photonics 7 , 93, (2013). 17 Ntziachristos, V. Going deeper than micros copy: the optical im aging frontier in biology. Nature Methods 7 , 603, (2010). 18 Čižmár, T., Ma zil u, M. & Dholakia, K. I n situ wavefront c orrection and its application to micromanipulation. Nat Photonics 4 , 388, (2010). 19 Horstmeyer, R., Ruan, H. & Yang, C. Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue. Nat Photonics 9 , 563, (2015). 20 Judkewitz, B., Wang, Y. M., Horstmeyer, R., Mathy, A. & Yang, C. Speckl e-scale focusing in the diffusive reg ime with time reversal of variance-encoded light (TROVE). Nat Photonics 7 , 300, (2013). 21 Si, K., Fiolka, R. & Cui, M. Fluorescence imaging be yond the ballistic regime b y ultrasound-pulse-guided digital phase conjugation. Nat Photonics 6 , 657, (2012). 22 Choi, Y. et al. Ov ercoming the Diff raction Limit Using Multiple L i ght Scattering in a Highly Disordered Medium. Physical Review Letters 107 , 023902, (2011). 23 Chaigne, T. et al. Controlling light in scattering media non-invasivel y us ing the photoacoustic transmission matrix. Nat Photonics 8 , 58, (2013). 24 Osnabrugge, G., Horstmeyer, R., Papadopoulos, I . N., Judkewitz, B. & Vellekoop, I. M. Generalized optical memory effect. Optica 4 , 886-892, (2017). 25 Freund, I., Rosenbluh, M. & Feng, S. Memor y Effects in Propagation of Optical 18 Waves through Disordered Media. Physical Review Letters 61 , 2328-2331, (1988). 26 Tang, D., Sahoo, S. K., Tran, V. & Dang, C. Sin gle-shot large field of view imaging with scattering media b y spatial demultiplexing. Applied Optics 57 , 7533-7538, (2018). 27 Sahoo, S. K., Tang, D. & Dang, C. Single -shot multispectral imaging with a monochromatic camera. Optica 4 , 1209-1213, (2017). 28 Edrei, E. & Scarcelli, G. Memory-effec t based deconvolution microscopy for super-resolution imag ing through sca tterin g media. Scientific reports 6 , 335 58, (2016). 29 Lyons, A. et al. Computational time-of-flight diffuse optical tomography. Nat Photonics , (2019). 30 Eggebrecht, A. T. et al. Mapping distributed br ain function and networ ks with diffuse optical tomography. Nat Photonics 8 , 448, (2014). 31 O’Toole, M., Lindell, D. B. & Wet zstein, G. Confocal non-line- of -sight imag in g based on the light-cone transform. Nature 555 , 338, (2018). 32 Bertolotti, J. et al. Non-invasive imaging throu gh opaque scattering layers. Nature 491 , 232-234, (2012). 33 Katz, O., Heidmann, P ., Fink, M. & Gigan, S. Non-invasive single-shot imaging through scattering la yers and around corners via speckle correlations. Nat Photon 8 , 784-790, (2014). 34 Okamoto, Y., Horisaki, R. & Tanida, J . Noninvasive three-dimensional i maging through scattering media b y three -dimensional sp eckle correlation. Opt Lett 44 , 2526- 2529, (2019). 35 Huang, B., Wang, W., Bates, M. & Zhuang, X. Three- Dimensional Super- Resolution I ma ging b y Stochastic Optical Reconstruction Microscopy. Science 319 , 810-813, (2008). 36 Rust, M. J ., Bates, M. & Zhuang, X. Sub-diff raction-limit imaging b y stochastic optical reconstruction microscopy (STORM). Nature Methods 3 , 793-796, (2006). 19 Figure 1. Super-resolution imaging through scatt ering media with SOSLI in comparison to oth er imaging techniques. (a) Schematic of SOS LI where incoherent light from blinking emitters hidden behind v arious scattering media is s cattered and then captured by a camera. (b & c) Ex perimental demonstrations of the cu rrent state - of-art invasive and non-invasive imaging, respe ctively, through scattering media in the identical experimental setup. (d) A simulation demonstration of super-resolution imaging reconstructed b y SOSLI. Simulation parameters are taken f rom the experiment in (b) and (c). Scale bars: 10 camera pix els, i.e. 65 µm on the imaging plane (see method). 20 Figure 2. Principle and simulation results of SOSLI. (a) Object constitutes man y intermittent emitters behind an optica l diffuser; th e iris defines the optical aperture of the imaging s y st em and the camera captures spec kle patterns. (b) A small portion of a typical speckle pattern. (c) Autocorrelation of the speckle pattern is similar to that of the emitter pattern. (d) A retrieved image from its autocorre lation. (e) Localized emitters from the retrieved image. (f) Estimated PSF’ from the localized emitter image (e) and its corresponding speckle p attern (b). (g) A serie s of localized emitter im ages by deconvolution of the speckle patte rns with the estimated PS F’. (h) A reconstructed image with a sub-diffraction-limit re solution by superposing all the individual localized emitter images. (i) A retrieved image from a si ngle-shot sp eckle pattern when all emitters are on, i.e. the current state- of -art non-in vasive imaging scheme 33 . 21 Figure 3. E xperimental results of imaging through a ground glass diffuser with different techniques. (a - b) Single-shot non-invasive imaging retrieved from the speckle autocorrelation and its intensity profile, respectively. (c - d) Invasive imaging by deconvolution of the single speckle pattern with an invasively me asured P SF and its intensity profile, respectivel y. (e - f) Non -invasive super-resolution imag ing by our SOSLI and its intensit y profile, re sp ectively. Three arrows on the left indicate the three lines for cross-sectional intensit y curves in figure b, d, f. Scale bars are 10 camera pixels, i.e. 65 µm on the imaging plane, and the values for X axis in gra phs are on the imaging plane (see method). 22 Figure 4. Experimental demonstration of thr ee t echniques for imagin g several complex objects hidden behind a ground glass diff user. (a - c) Our SOS LI approach for non-invasive super-resolution imaging. Insets are g round truth objects. (d - f) Non- invasive imaging r etrieved from autocorrelation of a single speckle p attern for the ground truth samples in the insets of a, b and c, respectivel y . (g - i) Invasive imaging with an invasivel y measured PSF and deconvoluti on approach for the ground truth samples in the insets of a, b and c, respectivel y. Scale bars: 10 camera pixels, i.e. 65 µm on the imaging plane (see method). 23 Figure 5. Experimental demonstration of non-invasive super-resolution imaging through a fresh chicken eggshell membrane by SOSLI . (a) Speckle correlation of PSFs at different me asurement time for the static scattering medium (ground glass) and the dy n amic one (fresh chicken eggshell membrane). (b) Th e reconstruc t ed image b y SOSLI with a single estimated PSF. (c) The reconstructed ima ge b y SOSLI with adaptive PSF estimation. Sc ale bars: 10 camera pixels, i.e. 65 µm on the imaging plan e (see method). 24 Supplementary material: Non-invasive super-resolution imaging through dynamic scattering media Dong Wang 1,2,# , Sujit K. Sahoo 1,3,# and Cuong Dan g 1,* 1 Centre for Optoelectronics and Biophotonics (COEB), S chool of Electrical and Electronic Engineering, Nanyang Technological Universit y Sin gapore, 50 Nanyang Avenue, 639798, Singapore 2 Key Laborator y o f Advanced Transducers and Intelligent Control Sy st em, Ministry of Education, and Shanxi Province, College of Physics and Optoelectronics, Taiyuan University of Technology , Tai yuan 030024, China 3 School of Electrical S ciences, Indian Institute of Technolo gy Go a, Goa 403401, India # Dong Wang and Sujit K. Sahoo contribute equally. * Corresponding author, E-mail: HCDang@ntu.edu.sg Optical experim ent setup The optical setup for experimental demonstration of stochastic optical scattering localization im aging (SOS LI) is depict ed sch ematically in Supplementary Fig. S1. It consists of two parts: the object simulator, and the imaging s etup. The former i s designed for convenient g eneration of v arious objec ts with blinking emitters. We replace the projection lens of a commercial proj ector ( Acer X1 13PH ) by a microscope objective (40x , numerical aperture: NA =0.65) to de-magnify th e projector pixels to       µm 2 squares at the object plane. Two irises , one placed in front of the projector and the other at the object plane, are used to block all the stray light generated by the projector. Light from the object passing through both scattering media and the imaging iris, is captured by a camera sensor ( Andor Neo 5.5, 2560×2160 pix els, and 6.5-µm pix el size) . The scattering medi a are a ground glass diffuser (a static one ) or fresh chicken e ggshell membrane (a d ynamic one) in our demonstration. An optical filter (Thorlabs FB550-10, 550 nm wavelength, and 10 nm full-width at half-maximum - FWHM) is mounted on the camera to narrow th e optical spectrum. Blinking emitters 25 are generated b y randoml y blinking p rojector pix els. For invasive measurement of the point spread function ( PSF ) , onl y one center pixel is turned on. Supplementary Fig. S1. O ptical setup t o demonstrate SOSLI for non-invasive super - resolu tion imaging t hr ough str ongly sc attering media. (a) Ob ject-simulator , which is designed for generating various microscop ic objects at the object plane. (b) Simple optical configuration for imaging setup where u = 10 m m and v = 100 mm . The state- of -th e-art non-invasive and in vasive imaging We conduct experiments for the demonstration of the current state- of -the-art non- invasive and invasive i maging through a 120-grit ground glass diffus er using the experimental setup shown in supplementary F i g. S1 . A non-invasive image is retrieved from the autocorrelation of a sing le-shot speckle p attern b y applying the phase retrieval algorithm. An invasive image is the deconvolution of a sin gle-shot speckle pattern with an invasivel y measured PSF. The diameters o f the imaging iris are set as 1 mm, 2 mm and 3 mm that cor respond to NAs of 0.05, 0.1 and 0.15, respectively, and t he diffraction limit of 6.7 µ m, 3.4 µm and 2.3 µm respectively . W e can easily see the effec ts of NA on resolution from the results given in supplementar y Fig. S 2. An im aging s ystem is a low pass filter, where higher NA (higher cut-off frequency) provides higher resolution (i.e. sharper) im ages tha n lower NA does. Because of the effects of noise , camera’s dynamic range and dark counts on the pe rformance of the phase re trieval al gorithm, the single-shot non -invasive im ages have a slightl y lower resolution than the diffraction limit. If these effects are too high, the algorithm may not even conver ge. On the other hand, the deconvolution images have a slightly h igher resolution than the diffraction limit because deconvolution recovers and enhances the high frequency components of images, i.e. sharper cut-off low pass filter. 26 Supplementary Fig. S2. The effect of NA on resolution for invasive and non-i nvasive imaging through scatter ing med ia. (a - c) The s peckle pat ter ns of the sam e object with N As of 0.05, 0.1, and 0.15, r especti vely . Experimental results of the state-of-the-art invasive (d - f) and non-inv asive (g)-(i) imag ing through the 120-grit ground g lass dif fuser with the speck le patterns in (a)- (c) respectively . Scale bar: 10 camera pixe ls , i.e. 65 µm on the i maging plane (see method ). Estimating a point spreading function from a rando mly selected stochastic emitter pattern From a series of stoch astic speckle patterns recorded for SOSLI, we pick up one pattern randomly (Fig. S3a). The iterative phase retrieval algorithm is utilized to re trieve the emitter pattern at low resolution as presented in Fig. S3b-c. Two similar emitter patterns shifted from each other c an be retrieved f rom a si ngle speckle pattern by two different runs of the al gorithm be cause autocorrelation onl y keeps the relative emitter positions while losing their exact positions. Figure S 3d-e present the emitter positions after localization. The localiz ed-emitter ima ges show ver y clean emitters, removing all the noise or artifacts of the phase retrieval al gorithm. Figure S3f-g show the estimated PS Fs calculated from a single spec kle pattern (Fig. S 3a) and two different phase 27 retrieval/localization results (Fig . S3d- e) . Figure S3i- j present the emitter positions localized after deconvolution of another speckle pattern (Fig. S3 h) with two estimated PSF s in Fig. S3f-g. The emitter positions in Fig. S3 i- j automatically ali gn ver y well with those in Fig. S3d-e , respectively because the y are reconstructed from the same estimated PS Fs. The absolute positions of emitters and PSF are not impo rtant in our SOSLI. They do not affect our results and usual imaging techniques do not concern about absolute position. Supplementary Fig. S3. Reconstruct ed and individual stochastic emitter pattern and estimated P SF for deconvolution. (a) A typical stochastic speckle pattern. (b - c) T ypi cal images retrieved from autocorrelation by phase retrieval al gorithm . (d - e) Localiz ed-emitter images. (f - g) Estimated PSFs. (h) Another st ochast ic speckle pattern. (i-k ) Emit ter positions calculated from a si ngle speck le pattern and two estimated PSFs. Scale bar: 200 c amera pixels for a, h, g, h and 1 0 camera pixels for b, c, d, e, i, j (see m ethod). Reconstruction with PSFs estimated fr om differ ent speckle patterns During o ur implementation of SOS LI , we randomly choose one pattern out of multiple collected speckle patterns for estimation of the PSF. Interestingly, different speckle patterns give us slightly d ifferent estimated PSFs. The differences a re not o nly arbitrary shift from each other but also the pattern itself (Supplementar y Fig. S 4a-c and S5b-c). However, the reconstruc t ed images from these estimated PSFs are very much simil ar 28 (Supplementary Fi g. S4 d-f) with different shifts . It seems that we onl y e stimate the main features of the PSF , which is suff icient for reconstruc tion. We test the similarity of these three estimated P SFs by calculating the correlation among them. The autocorrelation of PSF1 shows its random spe ckle nature with a bright spot (Gaussian profile) at the center (Supplementary F i g. S5a). The correlation of PSF1 with the other PSFs indicates an off-center bright spot with some background (Supplementary Fig. S5b- c) , impl ying that the se PSFs share the m ain features and shift from each other. The clear bright spot allows us to locate its center then obtain the relative shift between PSFs. The obtained relative shifts (Supplementary F ig. S5d) are exactl y equal to th e relative shifts between the retrieved objects (Supplementary Fig. S4 d-f ). Supplementary Fig. S4 . Retrieved results of an object w ith the PSFs estimated fro m differen t speckle patterns arbitrarily picked. (a - c) The estimated PSFs, (d - f) the corresponding reconstructe d imag es at supe r-resolution. Scale bar: 200 cam era pixels f or a-c, and 10 cam era pi xels for d- f (see method). The observation is interesting, important and useful. Because of the localization, we can easily remove the background artifact and noi se in the ph ase retrieval im ages and deconvolution images. Therefore, th e SOSLI appr oach can tolerate mo re error in PSF estimation. This no t only explains how and wh y the SOS LI should work v ery well in static scattering media, but also insp ires us to conceive a successful solution for dynamic scattering media in the section 7. 29 Supplementary Fig. S5 . Cross-corr elations of the PSF1 with the ot her PS Fs to obtain the relative shifts between them. (a - c) The correlation patterns. (d) The relativ e shifts between PSF1 with other PSFs . The num bers on the side of im age indicates the pixel num ber s. Reconstruction of super -r esolution images with differ ent numbers of stochastic p atterns. In our experiment, the object to be imaged through a diffuser constitutes mul tiple blinking emitters. SOSLI technique reconstructs a super -resolution image from multiple stochastic patterns. The qualit y of the reconstructed ima ge will increase with the number of frames. We c haracterize the reconstructed images with various numbers (n=200, 400, 800, and 8000) of the randomly blinking emitter patterns whi ch are used Fig. 3c. The results are sh own in supplementar y Fig . S6 where supplement ary Fig. S6e- h show the results of s upplementary Fi g. S6a -d aft er the bicubic interpolation processing. It is obvious that reconstruction with 8000 stocha stic patterns gives the be st image qualit y (supplementar y Fig. S6d&h). How ever, 300-400 frames are reasonabl y good for the reconstruction of our simple object (Supplementary Fig. S6b&f). 30 Supplementary Fig. S6. Retrieved r esults o f an objec t by SOSLI with the differ ent numbers of stochastic patterns . (a - d) T he raw res ults f rom SOSLI. (e - h) The results after bicubic in terpolation pro cessing. Scale bar: 10 c amera pixels, i.e. 65 µm on imaging plane (see method ). Deconvolution results with a decorre lated PSF Supplementary Fig. S7. Deconvolut ion resul ts with a decorr elated P SF of chicken eggshell membrane. a) T he first stochastic speckle pat tern is chosen t o estimate the PSF . b) The estimated PSF from the f irst stochastic speckle pattern. c-e) The stochastic sp eck le patterns with the mem brane’s decorrelation (the correlation coeff icients w it h respect to the pattern number are p resented in Fig . 5a: 0.65, 0.36, 0.2 respec tively). f -h ) The deconvolution results of the speckle patterns in c-e, respectively , with the esti mated PSF from the first speckle pattern. 31 Scale bar: 200 cam era pixels for a-e, and 10 c amera pixels for f-h (see method ). Adaptive appr oach for SOSLI to do super -resoluti on im ag ing thr ough a biologica l tissue Our adaptive approach for SOSLI need to define the number of stochastic frames for each section depending on the decorrelation time and ima ge acquisiti on time . The requirement is that every two adjacent sections h a ve some correlation. In our approach, we even divi de the set of patterns into overlapping se ctions (supplementary Fig. S8b) to guarantee correlation between adjacent sections . B y doing sectional reconstruction, we effectively op erate SOSLI with high speckle correlation for d ynamic scattering media (shading area in Fig. 5a and suppl ementary Fig. S8a). Individual super-resolution images are reconstructed for each section inde pendently. Before superposing these images on top of each other, we need to align them. As presented in the previous section, the relative shifts between the reconstruct ed images are equal to the r elative shifts between the estimated PSFs when the scattering media is not completely decorrelate d. W e calculate the correlation pattern of the estimated PSFs for every two adjac ent sections to identif y the r elative shifts between them. The vectors indicated the relative shifts between section  and section  as follows.   󰇍 󰇍 󰇍 󰇍 󰇍            󰇝      󰇞 where {} po siti on gives 2D coordinate of the bri ghtest spot center of an image, and  indicates the correlation calculation.  󰇝      󰇞 is simpl y the cent er of the image because of speckle-type PSF. Supplementary Fig. S9a-j show the 10 vectors   󰇍 󰇍 󰇍 󰇍 󰇍  for the frame set from 2 to 11 with respect to the first frame set. There relative positions are presented in supplementary Fig. S9k-l. With this, all the individual supe r - resolution images can be inversely shifted before s uperposing (supplementary Fig. S8c) to achieve the full image as presented in Fig. 5c. 32 Supplementary Fig. S8 . Schematic for t he implementation of SOSL I w ith adaptive PSFs for imaging thr ough a biological tissue. (a) The measured decorrelation characteristic of a chicken eggshell membrane. ( b) Sectioning and overlapping of the collected speckle patterns. (c) Implem entat ion of SOSLI with adaptive PSFs for r etrieving the hidden object at super- 33 resolution. Scale b ars: 200, 20 and 10 on the s cale bar are camera pixels (see m ethod). Supplementary Fig. S9 . The shifted vectors and the r elative position of sectional super - resolu tion i mages. (a-j) the shifted v ectors   󰇍 󰇍 󰇍 󰇍 󰇍  between two adjacent sections. (k) The relative positions of the 10 sectional reconstructed imag es (from 2 to 1 1) with respec t to the first one. (l) Mag nified center region of (k ). The most adaptive appr oach for SOSLI for h ighly dynam ic scattering media By using th e most a d aptive SOSLI approac h, we can align the emitter pa tterns even with decorrelation of two consecutive speckle pattern up to 80% (Fi g. S10). The important point is to find the center of the bri ghtest spot in correlation b etween two PSFs estimated from two consecutive spe ckle patterns. It might look similar to localizing the emitters from noisy deconvolution image due to 80% decorrelation in Fig. S7h. How ever, finding the brightest spot (there is only one) then localizing its center in Fig. S10h is relatively simpler than localizing all the centers of bright spot s in Fig . S7h while we do not know the number of emitters. 34 Supplementary Fig. S10 . The most adapt ive approach for SOSLI to mitigate highly dynamic scatter ing media. a -b) T wo speck le patterns taken when scattering media dec orrelate 80%, i.e. the correlat ion of 20%. c -d) Emitter patterns recovered from speckle patterns in a&b, re spectively , by the phase retrieval algorithm and localization. e- f) The estim ated PSFs from speckle patterns in a&b, respectively . g) The autoco rr elation of PSF1 i n figure e. h) The correlation betwe en PSF1 (in figure e) and PSF300 (in figure f). i) T he centers of brig htes t spots in g&h, showing the relative sh ift between two PSFs, which is also the r elative shift between two emitter patterns in c&d . Scal e bar: 200 camera pix els for a-b,e-f, and 10 camera pixels fo r c-d,g- i ( see method).