[OpenCV Getting Started Tutorial 9] Nonlinear filtering special session: median filtering, bilateral filtering[Copy link]
This post was last edited by Rambo on 2017-12-27 14:25
As we mentioned in the previous article, linear filtering can achieve many different image transformations. However, nonlinear filtering, such as median filters and bilateral filters, can sometimes achieve better results. Other examples of neighborhood operators include morphological operators that operate on binary images, semi-global operators for computing distance transforms and finding connectivity. Here is a screenshot:
:
1. Theory and Concept Explanation - From Phenomenon to Essence
1.1 Overview of Nonlinear Filtering In the previous article, the filters we considered were all linear, that is, the response of the sum of two signals is equal to the sum of their individual responses. In other words, the output value of each pixel is a weighted sum of some input pixels. Linear filters are easy to construct and easy to analyze from the perspective of frequency response. In fact, in many cases, using nonlinear filtering of neighboring pixels may produce better results. For example, when the noise is shot noise rather than Gaussian noise, that is, when the image occasionally has large values. In this case, if the image is blurred with a Gaussian filter, the noise pixels will not be removed, they will just be converted into softer but still visible shot. This is where the median filter comes in. 1.2 Median filter Median filter is a typical nonlinear filtering technology. The basic idea is to replace the gray value of the pixel with the median of the gray value of the pixel neighborhood. This method can remove impulse noise and salt and pepper noise while retaining the edge details of the image. Median filtering is a nonlinear signal processing technology based on sorting statistics theory that can effectively suppress noise. Its basic principle is to replace the value of a point in a digital image or digital sequence with the median of the values of each point in a neighborhood of the point, so that the surrounding pixel values are close to the true value, thereby eliminating isolated noise points. It is particularly useful for speckle noise and salt-and-pepper noise because it does not depend on the values in the neighborhood that are very different from the typical values. The median filter works similarly to the linear filter when processing continuous image window functions, but the filtering process is no longer a weighted operation. Median filtering can overcome the image detail blurring caused by common linear filters such as minimum mean square filtering, box filter, mean filtering, etc. under certain conditions, and is very effective in filtering out pulse interference and image scanning noise. It is also often used to protect edge information. The characteristic of preserving edges makes it useful in situations where edge blurring is not desired. It is a very classic method for smoothing noise.
●Comparison between median filter and mean filter
Advantages of median filter and mean filter: In mean filter, since the noise component is put into the average calculation, the output is affected by the noise, but in median filter, since the noise component is difficult to select, it will hardly affect the output. Therefore, when the same 3x3 area is used for processing, the median filter has a better ability to eliminate noise. Median filter is a good method for both eliminating noise and preserving edges.
Disadvantages of median filter and mean filter: Median filter takes more than 5 times the time of mean filter.
As the name implies, the median filter selects the median of the neighborhood pixels of each pixel as the output, or in other words, the median filter sets the grayscale value of each pixel to the median of the grayscale values of all pixels in a certain neighborhood window of the point. For example, take a 3 x 3 function window and calculate the median of the pixels in the function window centered at point [i,j] as follows:
(1) Arrange the pixels according to the intensity value.
(2) Select the middle value of the sorted pixel set as the new value of point [i,j]. This process is shown in the figure below. http://img.blog.csdn.net/20140408150815000?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcG9lbV9xaWFubW8=/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast The median is usually calculated using the neighborhood of an odd number of points, but if the number of pixels is even, the median is calculated using the neighborhood of an odd number of points. When is used, the median is the average of the two middle points of the sorted pixels. The results of using the median filter with different neighborhood sizes are shown in the figure. Under certain conditions, median filtering can overcome the blurred image details caused by linear filters (such as mean filtering), and is most effective in filtering out pulse interference, i.e. image scanning noise. In the actual operation process, the statistical characteristics of the image are not needed, which also brings a lot of convenience to the calculation. However, for some images with many details, especially lines, spires and other details, it is not suitable to use median filtering. 1.3 Bilateral filtering Bilateral filtering The filter is a nonlinear filtering method that is a compromise between the spatial proximity and pixel value similarity of the image, while taking into account the spatial information and grayscale similarity to achieve edge-preserving denoising. It is simple, non-iterative, and local. The advantage of bilateral filter is that it can preserve edges. Generally, the Wiener filter or Gaussian filter used in the past to reduce noise will blur the edges obviously, and the protection effect of high-frequency details is not obvious. As the name suggests, the bilateral filter has an additional Gaussian variance sigma-d compared to the Gaussian filter. It is a Gaussian filter function based on spatial distribution. Therefore, near the edge, the pixels far away will not affect the pixel values on the edge too much, thus ensuring the preservation of the pixel values near the edge. However, due to the preservation of too much high-frequency information, the bilateral filter cannot cleanly filter out the high-frequency noise in the color image, and can only filter the low-frequency information better. In the bilateral filter, the value of the output pixel depends on the weighted value combination of the neighboring pixel values:
The weighting coefficient w(i,j,k,l) depends on the product of the domain kernel and the range kernel. The domain kernel is expressed as follows (as shown in the figure):
Domain filtering corresponding icon:
The value range core is expressed as:
Value range filtering:
Multiplying the two together produces a data-dependent bilateral filter weight function:
2. In-depth - OpenCV source code analysis and tracing First, let us appreciate the source code of the medianBlur() function, which starts at line 1653 of ...\opencv\sources\modules\imgproc\src\smooth.cpp. [cpp]view plaincopy
//----------------------------------【medianBlur() function Chinese annotated version source code】-------------------------- // Code function: Function for median filtering operation // Description: The following code is from the official source code of the computer open source vision library OpenCV // OpenCV source code version: 2.4.8 // Source code path: …\opencv\sources\modules\imgproc\src\smooth.cpp // The starting line number of the following code in the source file: 1653 lines // Chinese comments by Qianmo //-------------------------------------------------------------------------------------------------------- void cv::medianBlur( InputArray _src0,OutputArray _dst, int ksize ) { //Copy parameter Mat data to temporary variables for later operations Mat src0 = _src0.getMat(); _dst.create( src0.size(), src0.type() ); Mat dst = _dst.getMat(); //Handle different situations for specific ksize values if( ksize <= 1 ) { src0.copyTo(dst); return; } CV_Assert( ksize% 2 == 1 ); //If the HAVE_TEGRA_OPTIMIZATION optimization option has been defined before, the tegra optimized version of the function in the macro body will be executed and returned #ifdef HAVE_TEGRA_OPTIMIZATION if (tegra::medianBlur(src0, dst, ksize)) return; #endif bool useSortNet = ksize == 3 || (ksize == 5 #if !CV_SSE2//If CV_SSE2 is false, execute the statements in the macro body&& src0.depth() > CV_8U #endif ); //Start the formal filtering operation Mat src; if( useSortNet ) { if( dst.data != src0.data ) src = src0; else src0.copyTo(src); //According to different bit depths, give different template type values to the medianBlur_SortNet functionif( src.depth() == CV_8U ) medianBlur_SortNetksize)) return; #endif bool useSortNet = ksize == 3 || (ksize == 5 #if !CV_SSE2//If CV_SSE2 is false, execute the statement in the macro body&& src0.depth() > CV_8U #endif ); //Start the formal filtering operation Mat src; if( useSortNet ) { if( dst.data != src0.data ) src = src0; else src0.copyTo(src); //According to different bit depths, give different template type values to the medianBlur_SortNet functionif( src.depth() == CV_8U ) medianBlur_SortNetksize)) return; #endif bool useSortNet = ksize == 3 || (ksize == 5 #if !CV_SSE2//If CV_SSE2 is false, execute the statement in the macro body&& src0.depth() > CV_8U #endif ); //Start the formal filtering operation Mat src; if( useSortNet ) { if( dst.data != src0.data ) src = src0; else src0.copyTo(src); //According to different bit depths, give different template type values to the medianBlur_SortNet functionif( src.depth() == CV_8U ) medianBlur_SortNet<minmax8u, minmaxvec8u="">( src, dst, ksize ); else if( src.depth() == CV_16U ) medianBlur_SortNet<minmax16u, minmaxvec16u="">( src, dst, ksize ); else if( src.depth() == CV_16S ) medianBlur_SortNet<minmax16s, minmaxvec16s="">( src, dst, ksize ); else if( src.depth() == CV_32F ) medianBlur_SortNet<minmax32f, minmaxvec32f="">( src, dst, ksize ); else CV_Error(CV_StsUnsupportedFormat, ""); return; } else { cv::copyMakeBorder( src0, src, 0, 0, ksize/2, ksize/2, BORDER_REPLICATE); int cn = src0 .channels(); CV_Assert( src.depth() == CV_8U && (cn == 1 || cn == 3 || cn ==4) ); double img_size_mp = (double)(src0.total())/(1 << 20); if( ksize <= 3 + (img_size_mp < 1 ? 12 : img_size_mp < 4 ? 6 :2)*(MEDIAN_HAVE_SIMD && checkHardwareSupport(CV_CPU_SSE2) ? 1 : 3)) medianBlur_8u_Om( src, dst, ksize ); else medianBlur_8u_O1( src, dst, ksize ); } }
复制代码
[size=14px ] Carefully reading the source code, we can find that when the filtering operation is officially started, we will give different template type values to the medianBlur_SortNet function template, or call medianBlur_8u_Om or medianBlur_8u_O1 to perform the operation according to the different bit depths of the image. As we just mentioned above, medianBlur_SortNet is a function template. Its source code starts at line 1439 of smooth.cpp. Since its function body is very long, we only post its function here. statement. [ font=Verdana, Geneva, Arial, Helvetica, sans-serif][cpp]view plaincopy[/ color]
template static void medianBlur_SortNet( constMat& _src, Mat& _dst, int m ); In addition, the source code of the bilateralFilter function is also quite lengthy, in the D:\Program Files\opencv\sources\modules\imgproc\src\smooth.cpp source code file. From line 1714 to line 2273. We only give the path here, and an overview diagram, everyone is interested in looking at the source code yourself. One more thing to mention, lines 2275 to 2552 of the smooth.cpp source code are the source code of the adaptive bilateral filter (adaptiveBilateralFilter) in OpenCV. Those who are interested and energetic can explore it. 3. Easy introduction - Quick introduction to API functions 3.1 Median filtering - medianBlur function The medianBlur function uses a median filter to smooth (blur) an image, input from src, and the result is output from dst. And for multi-channel images, each channel is processed separately, and supports in-place operation (In-placeoperation). [cpp]view plaincopy
C++: void medianBlur(InputArray src,OutputArray dst, int ksize)
Parameter details:
The first parameter, src of InputArray type, is the input parameter of the function and is filled with a 1-, 3-, or 4-channel Mat type image. When ksize is 3 or 5, the image depth must be one of CV_8U, CV_16U, or CV_32F, and for images with larger aperture sizes, it can only be CV_8U.
The second parameter, dst of OutputArray type, is the target image. The output parameter of the function needs to have the same size and type as the source image. We can use Mat::Clone to initialize the target image with the source image as a template.
The third parameter, ksize of int type, is the aperture linear size. Note that this parameter must be an odd number greater than 1, such as 3, 5, 7, 9 ... Calling example: [cpp] view plain[ /color] copy[/ color]
//Load the original image [* ]Mat image=imread("1.jpg");
//Perform median filtering Operations
Mat out;
medianBlur( image, out, 7);
The complete program built with the above three core codes Code: [cpp] view plain[ /color] copy[/ color]
//--------------------- --------------【Program Description】-------------------------------- -------------- // Description: [Example program for using the medianBlur function of the median filter] // OpenCV version used for development: 2.4.8 // Created on April 3, 2014 by 浅ink//----------------------------------------------- ------------------------------------------------- / /-----------------------------------[Header file contains part]-------- ------------------------------- // Description: Contains the header files that the program depends on //------ -------------------------------------------------- -------------------------------------- #include "opencv2/core/core.hpp" # include"opencv2/highgui/highgui.hpp" #include"opencv2/imgproc/imgproc.hpp" //--------------------------- --------[Namespace declaration part]------------------------------------ --- // Description: Contains the namespace used by the program //---------------------------------- -------------------------------------------------- ----------- using namespace cv; //-----------------------------------【main() function】------- ------------------------------------- // Description: The entry function of the console application. The procedure starts here //------------------------------------------- -------------------------------------------------- -- int main() { //Load the original image Mat image=imread("1.jpg"); //Create window namedWindow("Median filter [Original image]"); namedWindow("Median filter [Effect Image】"); //Show the original imageimshow("Median filter【Original image】",image ); //Perform median filtering operation Mat out; medianBlur( image, out, 7); //Display effect graph imshow("Median filtering [effect graph]" ,out ); waitKey(0 ); }
复制代码
Running effect graph (the linear size of the aperture is 7): src] Use a bilateral filter to process an image. The image is input from src and the result is output from dst. [cpp]view plaincopy
C++: void bilateralFilter(InputArray src, OutputArraydst, int d, double sigmaColor, double sigmaSpace, int borderType=BORDER_DEFAULT)
The first parameter, src of InputArray type, input image, that is, source image, needs to be 8-bit or floating-point single-channel or three-channel image.
The second parameter, dst of OutputArray type, that is, target image, needs to have the same size and type as the source image.
The third parameter, d of int type, indicates the diameter of each pixel neighborhood during the filtering process. If we set this value to a non-positive number, OpenCV will calculate it from the fifth parameter sigmaSpace.
The fourth parameter, sigmaColor of double type, is the sigma value of the color space filter. The larger the value of this parameter is, the wider the colors in the neighborhood of the pixel will be mixed together, resulting in a larger area of semi-equal color.
The fifth parameter, the sigma value of the filter in the sigmaSpace coordinate space of double type, is the annotation variance of the coordinate space. The larger its value is, the more distant pixels will affect each other, so that larger areas with sufficiently similar colors will obtain the same color. When d>0, d specifies the neighborhood size and is independent of sigmaSpace. Otherwise, d is proportional to sigmaSpace. , "][cpp][cpp][cpp][cpp]view plain [url=http://blog.csdn.Use a complete example program to get familiar with the bilateralFilter function: [cpp]view plaincopy
//-----------------------------------[Program Description]---------------------------------------------- // Description: [Example program for using bilateral filtering bilateralFilter function] // OpenCV version used for development: 2.4.8 // Created by 浅墨 on April 3, 2014//------------------------------------------------------------------------------------------------ //-----------------------------------[Header file inclusion part]--------------------------------------- // Description: Contains header files that the program depends on//---------------------------------------------------------------------------------------------- #include "opencv2/core/core.hpp" #include"opencv2/highgui/highgui.hpp" #include"opencv2/imgproc/imgproc.hpp" //-----------------------------------[Namespace declaration part]--------------------------------------- // Description: Contains the namespace used by the program//----------------------------------------------------------------------------------------------- using namespace cv; //-----------------------------------[main( ) function]------------------------------------------------------------ // Description: The entry function of the console application, our program starts here//----------------------------------------------------------------------------------------------- int main() { //Load the original image Mat image=imread("1.jpg"); //Create a window namedWindow("Bilateral filtering [original image]" ); namedWindow("Bilateral filtering [effect image]"); //Display the original image imshow("Bilateral filtering [original image]", image ); //Perform bilateral filtering operation Mat out; bilateralFilter( image, out, 25, 25*2, 25/2 ); //Display the effect map imshow("Bilateral filtering [effect map]" ,out ); waitKey(0 ); }
复制代码
Run the effect map: Fourth, comprehensive examples - familiar with actual combat Still, each article will be equipped with a detailed annotated blog post supporting example program, which presents the knowledge points introduced in this article to everyone in the form of code. In this sample program, you can use the track bar to control the parameter values of various filters (box filter, mean filter, Gaussian filter, median filter, bilateral filter). By sliding the scroll bar, you can control the blurriness of the image under various smoothing processes. It is quite playable. Without further ado, here is the code: [cpp]view plaincopy
//-----------------------------------[Program Description]---------------------------------------------- // Program name:: "[OpenCV Getting Started Tutorial 9] Nonlinear Filtering Special: Median Filter, Bilateral Filter" Blog supporting source code// IDE version used for development: Visual Studio 2010 // OpenCV version used for development: 2.4.8 // Created by Qianmo on April 8, 2014 //----------------------------------------------------------------------------------- //-----------------------------------[Header file includes part]--------------------------------------- // Description: Contains header files that the program depends on//---------------------------------------------------------------------------------------------- #include8 // April 8, 2014 Created by Qianmo//------------------------------------------------------------------------------------------------ //-----------------------------------[Header file inclusion part]--------------------------------------- // Description: Contains header files that the program depends on//---------------------------------------------------------------------------------------------- #include8 // April 8, 2014 Created by Qianmo//------------------------------------------------------------------------------------------------ //-----------------------------------[Header file inclusion part]--------------------------------------- // Description: Contains header files that the program depends on//---------------------------------------------------------------------------------------------- #include<opencv2 core="" core.hpp="">
#include<opencv2 highgui="" highgui.hpp="">
#include<opencv2 imgproc="" imgproc.hpp="">
#include<iostream>
//-----------------------------------[Namespace declaration part]--------------------------------------- // Description: Contains the namespace used by the program//----------------------------------------------------------------------------------------------- using namespace std; using namespace cv; //-----------------------------------[Global variable declaration part]-------------------------------------- // Description: Global variable declaration//----------------------------------------------------------------------------------------------- Mat g_srcImage,g_dstImage1,g_dstImage2,g_dstImage3,g_dstImage4,g_dstImage5; int g_nBoxFilterValue=6; //Box filter kernel value int g_nMeanBlurValue=10; //Mean filter kernel value int g_nGaussianBlurValue=6; //Gaussian filter kernel value int g_nMedianBlurValue=10; //Median filter parameter value int g_nBilateralFilterValue=10; //Bilateral filter parameter value//-----------------------------------[Global function declaration part]-------------------------------------- // Description: Global function declaration //----------------------------------------------------------------------------------------------- //Trackbar callback function static void on_BoxFilter(int, void *); //Box filter static void on_MeanBlur(int, void *); //Mean block filter static void on_GaussianBlur(int, void *); //Gaussian filter static void on_MedianBlur(int, void *); //Median filter static void on_BilateralFilter(int, void*); //Bilateral filter //-----------------------------------【main() function】-------------------------------------------- // Description: The entry function of the console application, our program starts here//----------------------------------------------------------------------------------------------- int main() { system("color 5E"); //Load the original image g_srcImage= imread( "1.jpg", 1 ); if(!g_srcImage.data ) { printf("Oh, no, error reading srcImage~! : \n"); return false; } //Clone the original image into four Mat typesg_dstImage1= g_srcImage.clone() ); g_dstImage2= g_srcImage.clone() ); g_dstImage3= g_srcImage.clone() ); g_dstImage4= g_srcImage.clone() ); g_dstImage5= g_srcImage.clone() ); //Display the original imagenamedWindow("【<0>Original image window】", 1); imshow("【<0>Original image window】",g_srcImage); //=================【<1>Box filter】========================== //Create windownamedWindow("【<1>Box filter】", 1); //Create trackbarcreateTrackbar("Kernel value:", "【<1>Box filter】",&g_nBoxFilterValue, 50,on_BoxFilter ); on_MeanBlur(g_nBoxFilterValue,0); imshow("【<1>Box filter】", g_dstImage1); //======================================================== //==================【<2>Mean filter】============================ //Create window namedWindow("【<2>Mean filter】", 1); //Create trackbar createTrackbar("Kernel value:", "【<2>Mean filter】",&g_nMeanBlurValue, 50,on_MeanBlur ); on_MeanBlur(g_nMeanBlurValue,0); //======================================================== //==================【<3>Gaussian filter】============================= //Create window namedWindow("【<3>Gaussian filter】", 1); //Create trackbar createTrackbar("Kernel value:", "【<3>Gaussian filter】",&g_nGaussianBlurValue, 50,on_GaussianBlur ); on_GaussianBlur(g_nGaussianBlurValue,0); //======================================================== //==================【<4>Median filter】============================== //Create window namedWindow("【<4>Median filter】", 1); //Create track bar createTrackbar("Parameter value:", "【<4>Median filter】",&g_nMedianBlurValue, 50,on_MedianBlur ); on_MedianBlur(g_nMedianBlurValue,0); //======================================================== //==================【<5> Bilateral filtering】=============================== //Create window namedWindow("【<5> Bilateral filtering】", 1); //Create track bar createTrackbar("Parameter value:", "【<5> Bilateral filtering】",&g_nBilateralFilterValue, 50,on_BilateralFilter); on_BilateralFilter(g_nBilateralFilterValue,0); //========================================================== // Output some help information cout<<endl<<"\t嗯。好了，请调整滚动条观察图像效果~\n\n" )="" )函数】------------------------------------="" *)="" -----------------------------------------------------------------------------------------------="" -----------------------------【on_boxfilter(="" -1,size(="" 0;="" <<"\n\n\t\t\t\tby浅墨";="" <<"\t按下“q”键时，程序退出~!\n"="" boxfilter(g_srcimage,="" g_dstimage1,="" g_nboxfiltervalue+1));="" g_nboxfiltervalue+1,="" imshow("【<1="" on_boxfilter(int,="" return="" static="" void="" while(char(waitkey(1))!="q" {="" {}="" }="" 描述：方框滤波操作的回调函数="" 方框滤波操作="" 显示窗口="">方框滤波】", g_dstImage1);
提升卡
变色卡
千斤顶
京公网安备 11010802033920号
