Image Filtering Using Convolution in OpenCV

Have you ever tried to blur or sharpen an image in Photoshop, or with the help of a mobile application? If yes, then you have already used convolution kernels. Here, we will explain how to use convolution in OpenCV for image filtering. 

You will use 2D-convolution kernels and the OpenCV Computer Vision library to apply different blurring and sharpening techniques to an image. We will show you how to implement these techniques, both in Python and C++.

1. An Introduction to Convolution Kernels in Image Processing
2. How to Use Kernels to Sharpen or Blur Images?
3. Applying Identity Kernel to an Image in OpenCV
4. Blurring an Image using a Custom 2D Convolution Kernel
5. Blurring an image using OpenCV’s Built-In Blurring Function
6. Applying Gaussian Blurring to an Image in OpenCV
7. Applying Median Blurring to an Image in OpenCV
8. Sharpening an Image using Custom 2D Convolution Kernel
9. Applying Bilateral Filtering to an Image in OpenCV
10. Interesting Applications
11. Summary

Let’s start by taking a look at the code that will be used to filter images. Each line of code will be discussed in detail so that you fully understand it.

Python

import cv2
import numpy as np

image = cv2.imread('test.jpg')

# Print error message if image is null
if image is None:
    print('Could not read image')

# Apply identity kernel
kernel1 = np.array([[0, 0, 0],
                    [0, 1, 0],
                    [0, 0, 0]])

identity = cv2.filter2D(src=image, ddepth=-1, kernel=kernel1)

cv2.imshow('Original', image)
cv2.imshow('Identity', identity)
    
cv2.waitKey()
cv2.imwrite('identity.jpg', identity)
cv2.destroyAllWindows()

# Apply blurring kernel
kernel2 = np.ones((5, 5), np.float32) / 25
img = cv2.filter2D(src=image, ddepth=-1, kernel=kernel2)

cv2.imshow('Original', image)
cv2.imshow('Kernel Blur', img)
    
cv2.waitKey()
cv2.imwrite('blur_kernel.jpg', img)
cv2.destroyAllWindows()

Click here to download the source code to this post

C++

// Import dependencies
#include <opencv2/opencv.hpp>
#include <iostream>

// Using namespaces to nullify use of c::function(); syntax and std::function(); syntax
using namespace std;
using namespace cv;

int main()
{
    // Read Image
    Mat image = imread("test.jpg");

    // Print Error message if image is null
    if (image.empty()) 
        {
            cout << "Could not read image" << endl;
        }
    
    // Apply identity filter using kernel
    Mat kernel1 = (Mat_<double>(3,3) << 0, 0, 0, 0, 1, 0, 0, 0, 0);
    Mat identity; 
    filter2D(image, identity, -1 , kernel1, Point(-1, -1), 0, 4);
    imshow("Original", image);
    imshow("Identity", identity);
    waitKey();
    imwrite("identity.jpg", identity);
    destroyAllWindows();

    // Blurred using kernel
    // Initialize matrix with all ones
    Mat kernel2 = Mat::ones(5,5, CV_64F);
    // Normalize the elements
    kernel2 = kernel2 / 25;
    Mat img;
    filter2D(image, img, -1 , kernel2, Point(-1, -1), 0, 4);
    imshow("Original", image);
    imshow("Kernel blur", img);
    imwrite("blur_kernel.jpg", img);
    waitKey();
    destroyAllWindows();
}

An Introduction to Convolution Kernels in Image Processing

In image processing, a convolution kernel is a 2D matrix that is used to filter images. Also known as a convolution matrix, a convolution kernel is typically a square, MxN matrix, where both M and N are odd integers (e.g. 3×3, 5×5, 7×7 etc.). See the 3×3 example matrix given below.

(1)  

A 3×3 2D convolution kernel

Such kernels can be used to perform mathematical operations on each pixel of an image to achieve a desired effect (like blurring or sharpening an image). But why would you want to blur an image? Here are two important reasons:

• Because it reduces certain types of noise in an image. For this reason, blurring is often referred to as smoothing.
• To remove a distracting background, you might intentionally blur portions of an image, as is done in ‘Portrait’ mode, on mobile device cameras.

Being a fundamental processing technique in Computer Vision, filtering images with kernels has many more applications.

How to Use Kernels to Sharpen or Blur Images

Filtering of a source image is achieved by convolving the kernel with the image. In simple terms, convolution of an image with a kernel represents a simple mathematical operation, between the kernel and its corresponding elements in the image.

• Assume that the center of the kernel is positioned over a specific pixel (p), in an image.
• Then multiply the value of each element in the kernel (1 in this case), with the corresponding pixel element (i.e. its pixel intensity) in the source image.
• Now, sum the result of those multiplications and compute the average.
• Finally, replace the value of pixel (p), with the average value you just computed.

Once you perform this operation for every pixel in the source image, using the above 3×3 kernel, the resulting filtered image will appear blurred. This is because the convolution operation with this kernel has an averaging effect, which tends to smooth or blur the image. You will soon see for yourself how the value of individual elements in a kernel dictate the nature of filtering. For example, by changing the value of the kernel elements, you can also achieve a sharpening effect. The concept is simple yet very powerful, and is therefore used in numerous image processing pipelines.

Now that you have learned to use convolution kernels, let’s explore how this is implemented in OpenCV.

You can also click here and visit the Colab notebook for this tutorial. You can run all the experiments there, without setting up your local system.

We will be using the following image for all our coding operations.

Applying the Identity Kernel to an Image in OpenCV

Before we describe how to implement blurring and sharpening kernels, let’s first learn about the identity kernel. The identity kernel is a square matrix, where the middle element is 1, and all other elements are zero, as shown below.

(2)  

A 3×3 identity kernel

What makes an identity matrix special is that multiplying it with any other matrix will return the original matrix. Let’s now demonstrate how to use this identity kernel with OpenCV filtering functions. In this first example, we will use the above identity kernel to show that the filtering operation leaves the original image unchanged.  

Start by importing OpenCV and Numpy, as shown in the code below.

Python

import cv2
import numpy as np

C++

// Import dependencies
#include <opencv2/opencv.hpp>
#include <iostream>

// Using namespaces to nullify use of c::function(); syntax and std::function(); syntax
using namespace std;
using namespace cv;

The following steps are performed in the code below:

• Read the test image
• Define the identity kernel, using a 3×3 NumPy array
• Use the filter2D() function in OpenCV to perform the linear filtering operation
• Display the original and filtered images, using imshow()
• Save the filtered image to disk, using imwrite()

filter2D(src, ddepth, kernel)

The filter2D()function requires three input arguments:

• The first argument is the source image
• The second argument is ddepth, which indicates the depth of the resulting image. A value of -1 indicates that the final image will also have the same depth as the source image
• The final input argument is the kernel, which we apply to the source image

Here is the code, both in Python and C++.

Python

image = cv2.imread('test.jpg')
"""
Apply identity kernel
"""
kernel1 = np.array([[0, 0, 0],
                    [0, 1, 0],
                    [0, 0, 0]])
# filter2D() function can be used to apply kernel to an image.
# Where ddepth is the desired depth of final image. ddepth is -1 if...
# ... depth is same as original or source image.
identity = cv2.filter2D(src=image, ddepth=-1, kernel=kernel1)

# We should get the same image
cv2.imshow('Original', image)
cv2.imshow('Identity', identity)

cv2.waitKey()
cv2.imwrite('identity.jpg', identity)
cv2.destroyAllWindows()

C++

// Apply identity filter using kernel
Mat kernel1 = (Mat_<double>(3,3) << 0, 0, 0, 0, 1, 0, 0, 0, 0);
Mat identity;
filter2D(image, identity, -1 , kernel1, Point(-1, -1), 0, 4);
imshow("Original", image);
imshow("Identity", identity);
waitKey();
imwrite("identity.jpg", identity);
destroyAllWindows();

As you can see in the below image, the filtered image (on the right) appears identical to the original image (on the left).

Blurring an Image using a Custom 2D-Convolution Kernel

Next, we will demonstrate how to blur an image. Here too, we will define a custom kernel, and use the filter2D() function in OpenCV to apply the filtering operation on the source image. 

Begin by defining a 5×5 kernel, consisting of only ones. Note that we also divide the kernel by 25. Why is that? Well, before you apply any convolution to an image, using a 2D-convolution matrix, you need to ensure that all the values are normalized. This is done by dividing each element of the kernel, by the number of elements in the kernel, which in this case is 25. This ensures all values stay within the range of [0,1]. 

Now use the filter2D() function to filter the image. As you can see, filter2D() can be used to convolve an image, with any user-defined kernel.

Python

"""
Apply blurring kernel
"""
kernel2 = np.ones((5, 5), np.float32) / 25
img = cv2.filter2D(src=image, ddepth=-1, kernel=kernel2)

cv2.imshow('Original', image)
cv2.imshow('Kernel Blur', img)

cv2.waitKey()
cv2.imwrite('blur_kernel.jpg', img)
cv2.destroyAllWindows()

C++

// Blurred using kernel
// Initialize matrix with all ones
Mat kernel2 = Mat::ones(5,5, CV_64F);

// Normalize the elements
kernel2 = kernel2 / 25;
Mat img;
filter2D(image, img, -1 , kernel2, Point(-1, -1), 0, 4);
imshow("Original", image);
imshow("Kernel blur", img);
imwrite("blur_kernel.jpg", img);
waitKey();
destroyAllWindows();

Look at the results in image given below and note how the filtered image (on the right) has been blurred compared to the original image (on  the left).

Blurring an Image Using OpenCV’s Built-In Function

You can also blur an image, using OpenCV’s built-in blur() function. Essentially a convenience function, use it to blur images, where you need not specifically define a kernel.  Simply specify the kernel size, using the ksize input argument, as shown in the code below. The blur function will then internally create a 5×5 blur kernel, and apply it to the source image.

The example below, which uses the blur() function will generate exactly the same output as the example above, which had used  the filter2d() function.

Python

"""
Apply blur using `blur()` function
"""
img_blur = cv2.blur(src=image, ksize=(5,5)) # Using the blur function to blur an image where ksize is the kernel size

# Display using cv2.imshow()
cv2.imshow('Original', image)
cv2.imshow('Blurred', img_blur)

cv2.waitKey()
cv2.imwrite('blur.jpg', img_blur)
cv2.destroyAllWindows()

C++

// Blurred using OpenCV C++ blur() function
Mat img_blur;
blur(image, img_blur, Size(5,5));
imshow("Original", image);
imshow("Blurred", img_blur);
imwrite("blur.jpg", img_blur);
waitKey();
destroyAllWindows();

Applying Gaussian Blurring to an Image in OpenCV

We will now apply a Gaussian blur to an image, using OpenCV. This technique uses a Gaussian filter, which performs a weighted average, as opposed to the uniform average described in the first example. In this case, the Gaussian blur weights pixel values, based on their distance from the center of the kernel. Pixels further from the center have less influence on the weighted average. The following code convolves an image, using the GaussianBlur() function in OpenCV.

GaussianBlur(src, ksize, sigmaX[, dst[, sigmaY[, borderType]]])

The GaussianBlur() function requires four input arguments:

• The first argument, src, specifies the source image that you want to filter.
• The second argument is ksize, which defines the size of the Gaussian kernel. Here, we are using a 5×5 kernel.
• The final two arguments are sigmaX and sigmaY, which are both set to 0. These are the Gaussian kernel standard deviations, in the X (horizontal) and Y (vertical) direction. The default setting of sigmaY is zero. If you simply  set sigmaX to zero, then the standard deviations are computed from the kernel size (width and height respectively). You can also explicitly set the size of each argument to positive values greater than zero.

Python

"""
Apply Gaussian blur
"""
# sigmaX is Gaussian Kernel standard deviation 
# ksize is kernel size
gaussian_blur = cv2.GaussianBlur(src=image, ksize=(5,5), \\
sigmaX=0, sigmaY=0)

cv2.imshow('Original', image)
cv2.imshow('Gaussian Blurred', gaussian_blur)
    
cv2.waitKey()
cv2.imwrite('gaussian_blur.jpg', gaussian_blur)
cv2.destroyAllWindows()

C++

// Performing Gaussian Blur
Mat gaussian_blur;
GaussianBlur(image, gaussian_blur, Size(5,5), SigmaX=0, SigmaY=0);
imshow("Original", image);
imshow("Gaussian Blurred", gaussian_blur);
imwrite("gaussian_blur.jpg", gaussian_blur);
waitKey();
destroyAllWindows();

The result is shown in figure given below. As you can see, there is a slight amount of blurring in the filtered image on the right.

Applying Median Blurring to an Image in OpenCV

We can also apply median blurring, using the medianBlur() function in OpenCV. In median blurring, each pixel in the source image is replaced by the median value of the image pixels in the kernel area.

medianBlur(src, ksize)

This function has just two required arguments:

• The first is the source image.
• The second is the kernel size, which must be an odd, positive integer.

Python

"""
Apply Median blur
"""
# medianBlur() is used to apply Median blur to image
# ksize is the kernel size
median = cv2.medianBlur(src=image, ksize=5)

cv2.imshow('Original', image)
cv2.imshow('Median Blurred', median)
    
cv2.waitKey()
cv2.imwrite('median_blur.jpg', median)
cv2.destroyAllWindows()

C++

// Apply Median Blur
Mat median_blurred;
medianBlur(image, median_blurred, (5,5));
imshow("Original", image);
imshow("Median Blurred", median_blurred);
imwrite("median_blur.jpg", median_blurred);
waitKey();
destroyAllWindows();

See the results of median blurring in figure given below. Note how for the same kernel size, the effect of median blurring is more prominent than Gaussian blurring. Median burring is often used to reduce ‘salt and pepper’ noise in images, as shown here.

Sharpening an Image Using Custom 2D-Convolution Kernels

You can also sharpen an image with a 2D-convolution kernel. First define a custom 2D kernel, and then use the filter2D() function to apply the convolution operation to the image.

In the code below, the 3×3 kernel defines a sharpening kernel. Check out this resource to learn more about commonly used kernels.

Python

"""
Apply sharpening using kernel
"""
kernel3 = np.array([[0, -1,  0],
                   [-1,  5, -1],
                    [0, -1,  0]])
sharp_img = cv2.filter2D(src=image, ddepth=-1, kernel=kernel3)

cv2.imshow('Original', image)
cv2.imshow('Sharpened', sharp_img)
    
cv2.waitKey()
cv2.imwrite('sharp_image.jpg', sharp_img)
cv2.destroyAllWindows()

C++

// Apply sharpening using kernel
Mat sharp_img;
Mat kernel3 = (Mat_<double>(3,3) << 0, -1,  0, 
                                   -1,  5, -1, 
                                    0, -1, 0);
filter2D(image, sharp_img, -1 , kernel3, Point(-1, -1), 0, BORDER_DEFAULT);
imshow("Original", image);
imshow("Sharpenned", sharp_img);
imwrite("sharp_image.jpg", sharp_img);
waitKey();
destroyAllWindows();

So, what kind of results do we get? Have a look at figure given below. The sharpening effects are very impressive. The sharpened image on the right reveals cracks in the wood that were not visible before.

Applying Bilateral Filtering to an Image in OpenCV

While blurring can be an effective way to reduce noise in an image, it is often not desirable to blur the entire image, as important details and sharp edges may be lost. In such cases, bilateral filtering can make your life easier.

• This technique applies the filter selectively to blur similar intensity pixels in a neighborhood. Sharp edges are preserved, wherever possible.
• It lets you control not only the spatial size of the filter, but also the degree to which the neighboring pixels are included in the filtered output. This is done, based on variation in their color intensity, and also distance from the filtered pixel.

Bilateral filtering essentially applies a 2D Gaussian (weighted) blur to the image, while also considering the variation in intensities of neighboring pixels to minimize the blurring near edges (which we wish to preserve). What this means is that the shape of the kernel actually depends on the local image content, at every pixel location.

Here’s a concrete example. Assume, you are filtering a region in an image, near an edge. A simple Gaussian blur filter would blur the edge because it lies near the filtered region (close to the center of the Gaussian filter).  But the bilateral filter can sense the edge, because it also considers differences in pixel intensities. So, it will compute a much lower weight for the pixels straddling the edge, thereby reducing their influence on the filtered region. Regions of more uniform intensity are blurred heavier, as they are not associated with strong edges.

Thankfully, OpenCV provides the bilateralFilter() function to filter images.

bilateralFilter(src, d, sigmaColor, sigmaSpace)

This function has four required arguments:

• The first argument of the function is the source image.
• The next argument d, defines the diameter of the pixel neighborhood used for filtering.
• The next two arguments, sigmaColor and sigmaSpace define the standard deviation of the (1D) color-intensity distribution and (2D) spatial distribution respectively.
  • The sigmaSpace parameter defines the spatial extent of the kernel, in both the x and y directions (just like the Gaussian blur filter previously described).
  • The sigmaColor parameter defines the one-dimensional Gaussian distribution, which specifies the degree to which differences in pixel intensity can be tolerated.

The final (weighted) value for a pixel in the filtered image is a product of its spatial and intensity weight. Thus,

• pixels that are similar and near the filtered pixel will have influence
• pixels that are far away from the filtered pixel will have little influence (due to the spatial Gaussian)
• pixels that have dissimilar intensities will have little influence (due to the color-intensity Gaussian), even if they are close to the center of the kernel.

Python

"""
Apply Bilateral Filtering
"""
# Using the function bilateralFilter() where d is diameter of each...
# ...pixel neighborhood that is used during filtering.
# sigmaColor is used to filter sigma in the color space.
# sigmaSpace is used to filter sigma in the coordinate space.
bilateral_filter = cv2.bilateralFilter(src=image, d=9, sigmaColor=75, sigmaSpace=75)

cv2.imshow('Original', image)
cv2.imshow('Bilateral Filtering', bilateral_filter)

cv2.waitKey(0)
cv2.imwrite('bilateral_filtering.jpg', bilateral_filter)
cv2.destroyAllWindows()

C++

// Apply bilateral filtering
Mat bilateral_filter;
bilateralFilter(image, bilateral_filter, 9, 75, 75);
imshow("Original", image);
imshow("Bilateral filtering", bilateral_filter);
imwrite("bilateral_filtering.jpg", bilateral_filter);
waitKey(0);
destroyAllWindows();
return 0;

Check out the results of bilateral filtering in figure given below. See how regions of more uniform pixel intensity have been smoothed (blurred), while preserving the fine cracks (edges) in the wood. Bilateral filtering is a very effective technique, but can prove computationally  expensive (especially for large kernel sizes). So choose wisely, depending on your particular application.

Summary

We started with the concept of convolution kernels and how they can be used to filter images.  Then learned how they can be used to perform mathematical operations on each pixel of an image to achieve a desired effect, such as blurring or sharpening. Saw how to implement 2D filtering, using OpenCV. After understanding  the identity kernel, we went on to create more custom kernels that can be used with the filter2D() function in OpenCV. Explored some important built-in filtering functions in OpenCV like MedianBlur() and GaussianBlur(). Finally, demonstrating the bilateralFilter()in OpenCV, seeing how it smoothens an image while maintaining crisp edges.