摘要
结构光照明显微成像(SIM)技术是一种在超分辨显微成像领域极具代表性的技术。尽管结构光照明能够提升空间分辨率,但其在实现超分辨过程中需要采集多幅图像,而且,诸如样本厚度不一、结构光平移的系统误差及环境噪声等因素均可能会对SIM成像质量产生影响。为解决这些问题,神经网络技术被引入SIM图像处理中。目前,常用的算法模型有卷积神经网络(CNN)及其变体以及生成对抗网络(GAN),它们通过训练样本来分析和纠正上述系统误差,提高时间分辨率,实现快速超分辨成像。据查,目前尚未有文章对于基于深度学习算法的结构光照明显微技术进行综合分析。鉴于此,本文介绍了SIM的基础原理并分析了深度学习算法近年来在提高SIM系统性能方面的应用,并对SIM技术未来的发展方向和面临的挑战进行了前瞻性探讨。
Significance Structured illumination microscopy(SIM)is a pivotal technique in super-resolution microscopy as it offers an innovative approach to enhance the spatial resolution exceedingly beyond that achievable by conventional optical microscopes.SIM harnesses the principle of structured illumination,where finely patterned light interacts with the specimen,thereby generating moiréfringes containing high-frequency information that is otherwise unaccessible owing to the diffraction limit.Achieving genuine super-resolution via SIM is involves intricate steps,including capturing numerous low-resolution images under an array of varied illumination patterns.Each of these images encapsulates a unique set of moirépatterns,which serve as the foundation for the subsequent computational reconstruction of a high-resolution image. Although effective, this methodology presentssome challenges. Biological samples, owing to their inherent irregularities and varying tissue thicknesses, can result in considerablevariability in the quality and consistency of the captured moiré patterns. This variability hinders the accurate reconstruction of highresolutionimages. Additionally, systematic errors can further complicate the process, thus potentially introducing artifacts or resultingin the loss of crucial details in the final image.Furthermore, sample damage due to prolonged light exposure must be considered when acquiring multiple images. Hence, thenumber of images required must be minimized without compromising the quality of the super-resolution reconstruction. Determiningthe optimal balance between the number of images and the quality of the final image is key in applying SIM to sensitive biologicalsamples.Image-processing algorithms are widely employed to mitigate the effect of excessive image pairs on imaging results. In addition tothe classical algorithms, recently developed deep-learning algorithms offer promising solutions. Deep-learning algorithms can extractmeaningful information from limited data and efficiently reconstruct images using neural networks. This approach enables high-qualitysuper-resolution images to be acquired faster without necessitating numerous input images. Consequently, in SIM imagereconstruction, satisfactory results can be achieved using fewer input images. Furthermore, deep-learning algorithms can effectivelymanage irregularities and variations in samples. By learning the structure and features of samples, these algorithms can better adapt todifferent types of samples, thus improving the robustness and accuracy of image reconstruction. This is particularly important whenmanaging complex biological samples, which typically exhibit diversity and variability. Therefore, analyzing and summarizing theapplications and effectiveness of deep learning in SIM systems is crucial.Progress In deep learning, the widely recognized efficient neural network models include the convolutional neural network (CNN),U-Net, and generative adversarial network (GAN). The CNN, which is renowned for its capacity to automatically discern patternsand features within intricate datasets, is particularly suitable for the task mentioned above. By undergoing rigorous training on asubstantial corpus of SIM images, the CNN learns to infer missing information that would otherwise require an array of supplementaryimages to capture. This predictive prowess enables the algorithm to amend the aberrations induced by SIM mode adjustments, thussignificantly improving the quality of the reconstructed images. Because of the strategic deployment of skip connections within U-Net,which ingeniously amalgamates information from both the deeper and shallower layers, the network can effectively preserve abundantdetails and information throughout the upsampling phase. Furthermore, the integration of deconvolution processes not only amplifiesthe dimensions of the output image but is also pivotal in enhancing U-Net’s exceptional performance and widespread acceptance withinthe biomedical sector. In the context of SIM reconstruction, harnessing U-Net to extract supplementary insights from available imagesallows the algorithm to construct high-resolution images from a minimal subset of input images, thereby considerably diminishing thelikelihood of specimen damage. By employing U-Net, one can reconstruct a super-resolved image similar to those afforded by classicalalgorithms using only three captured images. Furthermore, the implementation of GANs has significantly augmented the capabilitiesof deep-learning algorithms in SIM image processing. GANs comprise two dueling neural networks—a generator and a discriminator—that operate in tandem to fabricate highly realistic images. The generator synthesizes the images, whereas the discriminator assessestheir veracity. Similar to U-Net, GANs can reconstruct super-resolved images from three original images. However, GANs cangenerate data through adversarial learning, and when coupled with other architectures, they can achieve even better results.In summary, to enhance performance and generate high-resolution images from a minimal number of original images, variousneural network models are synergistically combined. Finally, the application of deep learning in nonstriped and non-super-resolutionSIM yields encouraging results, thereby further expanding the possibility of its applicability.Conclusions and Prospects The integration of deep-learning algorithms into SIM image processing significantly advances themicroscopy field. It not only addresses the technical challenges associated with achieving super-resolution but also provides newpossibilities for investigating the nanoscale world with unprecedented clarity and detail. As deep-learning algorithms continue toadvance, we expect more sophisticated algorithms to emerge and thus transcend the current boundaries of super-resolution microscopy.
作者
黎昕然
陈嘉杰
王美婷
郑晓敏
杜鹏
钟义立
戴小祺
屈军乐
邵永红
Li Xinran;Chen Jiajie;Wang Meiting;Zheng Xiaomin;Du Peng;Zhong Yili;Dai Xiaoqi;Qu Junle;Shao Yonghong(College of Physics and Optoelectronic Engineering,Key Laboratory of Optoelectronic Devices and Systems of Ministry of Education and Guangdong Province,Shenzhen University,Shenzhen 518060,Guangdong,China;College of Biomedical Engineering,Department of Medicine,State-Local Joint Engineering Laboratory of Key Technology of Medical Ultrasound,Key Laboratory of Biomedical Information Detection and Ultrasound Imaging and Guangdong Province,Shenzhen University,Shenzhen 518060,Guangdong,China)
出处
《中国激光》
CSCD
北大核心
2024年第21期35-49,共15页
Chinese Journal of Lasers
基金
国家自然科学基金(62275168,62275164,61775148,62204253,61905145)
国家重点研发计划(2022YFA1206300)
广东省自然科学基金(2021A1515011916、2023A1515012250)
广东省重大人才工程引进类项目(2021QN02Y124)
广东省教育厅重点专项(2023ZDZX2052)
深圳市科技计划项目(JCYJ20230808104901003,JCYJ20200109105608771)
深圳市光子学与生物光子学重点实验室项目(ZDSYS20210623092006020)
深圳大学医工交叉研究基金(2023YG002)
深圳大学科研仪器研制培育项目(2023YQ008)。
关键词
结构化照明显微系统
卷积神经网络
超分辨显微成像
图像处理
structured illumination microsystem
convolutional neural network
super-resolution microscopic imaging
image processing
