摘要
目的图像去雨技术是对雨天拍摄图像中雨纹信息进行检测和去除,恢复目标场景的细节信息,从而获得清晰的无雨图像。针对现有方法对雨纹信息检测不完全、去除不彻底的问题,提出一种联合自适应形态学滤波和多尺度卷积稀疏编码(multi-scale convolution sparse coding,MS-CSC)的单幅图像去雨方法。方法考虑雨纹信息的形状结构特点,构造一种自适应形态学滤波器来滤除有雨图像中的雨纹信息,获得包含图像自身纹理的低频成分;利用全变分模型正则化方法来增强低频成分的纹理信息,并利用有雨图像减去低频成分获得包含雨纹信息的高频成分;针对高频成分,根据雨纹的方向性提出一种基于方向梯度正则化的MS-CSC方法来重构高频成分,并通过迭代求解获得包含精确雨纹的高频成分,即雨层;利用有雨图像减去雨层得到最终的去雨图像。结果为验证本文方法的有效性,与一些主流的去雨方法进行实验比较。实验结果表明,本文方法在模拟数据集上的平均峰值信噪比(peak signal-to-noise ratio,PSNR)和平均结构相似度(structural similarity,SSIM)指标分别提高了0.95和0.0052,能得到较好的主观视觉效果和客观评价,特别是在真实数据集上具有明显优势。结论本文方法可有效去除雨纹,同时可保留更多的原始图像纹理信息,经证明是一种有效的去雨算法。
ObjectiveImages captured in rainy days are blurred due to the influence of raindrops or rain lines,which directly affects the accuracy of outdoor vision tasks,such as image segmentation,object recognition,and object tracking.Image deraining technology aims to recover the lost details in the target scene by detecting and removing the rain streak information in the rain image.According to the imaging principle of the image,the rain streak information in the image can be regarded as a kind of additive noise.Many traditional methods can use some prior information such as the direction,morphological structure and color of rain lines to remove rain noise and obtain a background layer without rain.However,due to the incomplete detection of rain streak information,many methods cannot completely remove rain streaks,resulting in residual rain streaks in the rain removal results.MethodBased on the existing problems,we propose a rain removal method based on adaptive morphological filtering and multi-scale convolutional sparse coding.First,according to the diversity of the width and length of the rain streak structure,a set of multi-scale structuring element operators is constructed by using the dilation operation in the morphology to filter out the rain streak information in the image.The size of the filter operator determines the degree of filtering.If the size of the filter operator is too large,more information will be filtered out,resulting in the loss of image texture.On the contrary,if the size of the filter operator is too small,the rain streak information cannot be completely filter out,resulting in the residual rain streak information in the image.Therefore,in order to control the degree of morphological filtering and avoid filtering out too much texture information of the image itself,an adaptive morphological filtering method is proposed.The method controls the size of the filter operator by comparing the wavelet filtering results with the morphological filtering results,so as to obtain low-frequency components with less rain streak information and rich texture information.Next,this method aims to preserve the texture information of the image itself in the low-frequency component as much as possible.Therefore,according considering the inevitable loss of image details in image filtering,the steerable total variation(STV)regularization method is used to detect the texture information of the image itself filtered out in the filtering,and supplement it to the low-frequency components.to obtain good deraining results,detecting accurate rain layer information is the key to deraining methods.Based on the above steps,a high frequency component(i.e.,rain layer)containing rain streak information the rain layer can be obtained by subtracting the detected low frequency component from the rain image.Then,to further refine the rain layer,we propose a high-frequency separation method based on multiscale convolution sparse coding(MS-CSC)to reduce the non-rain streak information in the high-frequency components and obtain a more accurate rain layer.In this process,in order to better detect the rain pattern information,by analyzing the falling direction of rain,two gradient constraints for the horizontal directionxand the vertical directionyare constructed to constrain the solution of the rain layer.The rain grain gradient information mainly exists in thexdirection,and the gradient information in theydirection is sparse.Therefore,because the gradient information of the rain streaks in thexdirection is relatively rich,this paper constructs the gradient sparse term in thexdirection by using the difference between the high frequency component and the reconstructed rain layer to ensure the rain streak reconstruction in the main directiony.Since the rain streak gradient information in theydirection is sparse,the gradient sparse term of the rain layer in theydirection is directly constructed to constrain the rain pattern information in thexdirection.By combining the CSC method and the constructed two gradient constraints,a directional gradient-regularized MS-CSC model is constructed to solve for an accurate rain layer.In this model,a dictionary needs to be constructed to implement the encoding of the rain layer.According to the diversity of rain streak structures in image scenes,this paper constructs a multi-scale dictionary to encode rain streak information of different sizes by using dictionaries of different sizes.Finally,the image de-rained result(i.e.,the background layer)is obtained by subtracting the rain layer from the rain image.ResultTo verify the effectiveness of the proposed method,we compared it with some state-of-the-art deraining methods,including some traditional and deep learning-based methods.Our demonstrated results illustrate that the proposed average peak signal-to-noise ratio(PSNR)and structural similarity(SSIM)index is improved by 0.95 and 0.0052 based on the simulated dataset.Moreover,our simulation method has its priority of qualitative visual effects and quantitative evaluation indexes.ConclusionThe integrated MS-CSC and adaptive morphological filtering method can remove rain streaks effectively.The recovery image has its clearer edge texture and details after rain removal.Additionally,our deraining algorithm method has strong generalization capability.
作者
黄淑英
许亚婷
杨勇
管巨伟
Huang Shuying;Xu Yating;Yang Yong;Guan Juwei(School of Computer Science and Technology,Tiangong University,Tianjin 300387,China;School of Software and Internet of Things Engineering,Jiangxi University of Finance and Economics,Nanchang 330032,China;School of Information Technology,Jiangxi University of Finance and Economics,Nanchang 330032,China)
出处
《中国图象图形学报》
CSCD
北大核心
2022年第5期1522-1536,共15页
Journal of Image and Graphics
基金
国家自然科学基金项目(61862030,62072218)
江西省自然科学基金项目(20192ACB20002,20192ACBL21008)。
