Image Enhancement Video Processing System Based on FPGA

Image enhancement processing is highly targeted and there is no unified evaluation standard. From the perspective of general picture and video appreciation, processing such as noise filtering, contrast expansion, sharpening and color enhancement can significantly improve the visual effect.

Here we design a real-time video image processing system based on FPGA, which includes two processing methods: contrast enhancement and color saturation. Compared with DSP and ASIC solutions, FPGA has absolute advantages in performance and flexibility, and it is more common to use FPGA to design video communication systems.

1 Principle and Algorithm

Image enhancement processing can be performed in the frequency domain and the spatial domain. Typical frequency domain methods, such as histogram enhancement processing, are suitable for software system implementation; while hardware systems are more suitable for spatial domain processing. Therefore, the processing methods described in this article will all be performed in the spatial domain.

1.1 Contrast expansion

Contrast expansion is also called grayscale transformation. Its purpose is to expand the grayscale range of interest while compressing the grayscale range of no interest. The simplest and most effective method is linear transformation, which satisfies the following relationship:

f(x, y) and g(x, y) represent the grayscale values ​​of the input image and the output image, respectively. After the transformation, the grayscale values ​​within a~b are linearly stretched, and the grayscale intervals of [0, a] and [b, 255] are suppressed. From the perspective of display devices, general civilian-grade displays do not have the ability to perfectly display 256 grayscales. Therefore, suppressing the grayscale intervals of too dark [0, a] and too bright [b, 255] and enhancing the dynamic range of the middle area can avoid the waste of grayscale. From another perspective, usually an image contains only a few pixels that are too dark and too bright. Purposefully and targetedly expanding the grayscale in the middle range and compressing the grayscale at both ends can enhance the image quality and obtain better visual effects, while the loss of image information is very small.

1.2 Color Enhancement

The purpose of color enhancement is to increase the color saturation of the image in a targeted manner while ensuring that the color is not distorted, making it look brighter and more vivid, with a stronger sense of layering.

1.2.1 Introduction to HSI Model

In color image processing, RGB, YCbCr, CMYK, etc. are commonly used color models. Their algorithms and conversions between them are easy to implement with hardware, but they are not well adapted to the colors actually interpreted from a human perspective.

Research shows that when observing a colored object from a human perspective, the object is generally described using three parameters: hue, color saturation, and brightness. Hue describes the properties of pure color, while saturation gives a measure of the degree to which a pure color is diluted by white light. Brightness is the degree of lightness or darkness of an image and is a subjective descriptive quantity. The HSI color model based on these three parameters is an ideal tool for developing image processing methods based on color description. The following briefly explains the principle of the HSI model.

Figure 1 shows a cube model of RGB color space, with the side length normalized to 1, the origin is black, and the opposite vertex is white. Connecting the black and white points gives the gray axis, and the saturation on this axis is 0, that is, there is no color component. Points with the same projection point on the gray axis have the same brightness, that is, points in the plane perpendicular to the gray axis have the same brightness value.

Take any point P in the cube, it defines a plane with the gray axis. According to the theory of color science, all colors are generated by the three colors located in the triangle defined by those colors. In this plane, the three vertices of the triangle are black, white and the color of P, and black and white cannot change the hue, so the points in this plane have the same hue as point P, that is, the isochromatic plane. Intuitively speaking, the closer the point is to the gray axis, the lighter the color, so the definition of color saturation is the distance between the point and the gray axis: the farther the distance, the stronger the saturation; the closer the distance, the weaker the saturation; if the distance is 0, the saturation is also 0, and there is no color at all.

In fact, the HSI space (isoluminance surface) is represented by the trajectory of colored points in a plane perpendicular to the grayscale axis. When the plane moves up and down along the grayscale axis, the boundary determined by the cross section formed by cutting the cube boundary is triangular or hexagonal. Here, a hexagon is taken as an example, as shown in Figure 2.

As shown in Figure 2, the three primary colors are divided by 120°. Cyan, magenta and yellow are called secondary colors, which are also divided by 120°. The primary and secondary colors are separated by 60°. For any given point Q in the figure, if the red axis is used as a reference, the angle H between the Q vector and the red axis determines its hue, and the vector length S determines its saturation. The position of the entire plane on the grayscale axis determines the brightness I of all points in the plane. So the conversion relationship from RGB to HSI is obtained:

1.2.2 Color saturation enhancement algorithm

The HSI model can easily adjust the hue and saturation, but its calculation is relatively complex and difficult to implement with hardware. However, according to its principle, the color saturation can be adjusted directly in the RGB space. Here, assuming that any point P (r, g, b) in the RGB cube, it is easy to find its projection point P* on the grayscale axis

, connecting P and P*, which is an isochromatic line, as shown in Figure 3.

As long as a suitable point (such as P1 or P2) is found on the extension line of P*P, the saturation of point P can be enhanced. Since the coordinates of P and P* are known, the equation of the straight line P*P can be obtained:

Let the value of equation (6) be t, and the parametric equation of the straight line P*P can be obtained:

The adjustment of color saturation can be achieved by adjusting the value of t. When t∈(-1,0), the point obtained is between P* and P, and the saturation is reduced; when t>0, the point obtained is outside P*P, and the saturation is enhanced.

2 Design ideas

2.1 Contrast expansion

The efficiency of implementing floating-point operations in hardware is low. Here, a table lookup method is used to perform grayscale transformation in the YCbCr space, as shown in Figure 4.

2.2 Color saturation enhancement

The color saturation adjustment is performed in the RGB space and is designed as a pipeline operation, as shown in FIG5 .

Color saturation enhancement is targeted, and different processing is required for pixels with different color saturation. The color saturation of an image can be divided into four levels, and pixels with low color saturation are enhanced, while pixels with high saturation are not processed or even suppressed. Figure 5 shows the pipeline operation of two-level color saturation adjustment: the first level sets t=1, and if the operation result overflows, it will be transferred to the second level adjustment (t=0.5). If the operation result still overflows, the output will keep the original input value (iR, iC, iB). The pipeline operation allows the saturation adjustment of each pixel to be completed in only one clock cycle on average, but the input has a delay of 6 clock cycles relative to the output. In order to achieve better results, the number of levels of saturation adjustment operations can be increased, but the consequence is that more hardware resources need to be occupied and longer delays are brought.

3 System Structure

The experimental platform is the DE2 development board of Ahera. The system structure is shown in Figure 6.

4 Test Results

The simulation result of the grayscale conversion module using ModelSim is shown in Figure 7. The simulation result of the color saturation enhancement module is shown in Figure 8.

The video image test comparison results are shown in Figures 9 and 10.

5 Conclusion

The experimental results show that this method can effectively improve image quality and enhance visual effects. Due to the characteristics of the algorithm, it is difficult to achieve real-time processing of color saturation enhancement using hardware systems. Starting from the basic principle, a relatively simple color saturation enhancement method that can be performed in RGB space is found. While meeting the real-time requirements, it can effectively achieve the purpose of color enhancement. However, due to the real-time requirements of video image processing and the characteristics of hardware system implementation, this method pays more attention to efficiency, and some details are not perfect, such as color distortion and other problems. Further research may focus on how to optimize hardware configuration and solve the problem of floating-point operations in the algorithm.