Design of video moving vehicle detection system based on DM6446

Abstract: This paper presents a technical implementation method of a video motion vehicle detection system based on the DM6446 processor, and introduces the hardware structure and software design ideas of the system. Combined with the characteristics of the DM6446 dual-core processor, the specific methods of communication between the processor ARM and DSP and double buffer switching are given. In the video motion vehicle detection algorithm, the difference accumulation method is used to model the background of the traffic video data acquired in real time by DM6446, and the background difference method is used to detect the vehicle motion area. Combined with the Otsu thresholding, morphological filtering and region growing algorithms, the real-time detection of video motion vehicles is finally realized on the DM6446 hardware platform. The experimental results show that the system has a good detection effect.

With the rapid development of modern transportation industry, the number of vehicles on the road is increasing day by day, and the accompanying problems such as traffic congestion and low road efficiency have brought many difficulties to daily traffic management. The intelligent transportation system (ITS) that integrates modern high-tech such as computers and electronics provides a solution.

Moving vehicle detection is an important part of ITS. This paper discusses the design of a traffic video information acquisition and processing system based on TI's TMS320DM6446 (DM6446 for short) as an embedded development platform. By analyzing real-time traffic video sequences, using difference accumulation background modeling, Otsu automatic threshold selection, morphological filtering and regional growth positioning techniques, the detection of moving vehicles in traffic scene videos is finally achieved.

1 System Hardware Composition

The TMS320DM6446 used in this system is a highly integrated digital media processor based on the two cores of ARM926 and TMS320C64x+. ARM926EJ-S uses pipelined pipelines to execute 32bit/16bit instruction sets, and provides independent 16KB instruction cache and 8KB data cache, which can effectively control and manage external interrupts, various interfaces and peripherals. TMS320C64x+ belongs to the TMS320C6000 series of high-performance fixed-point DSP processors, integrating 64 32-bit general registers and 8 functional units, hardware support block loop operation, and adopts a two-level cache structure: L1 program/data cache and L2 storage/cache. ARM and DSP share an external 256MB DDR SDRAM memory, and 64MB NAND Flash is used to store ARM and DSP codes.

The system hardware platform also includes a video processing subsystem (VPSS) and numerous peripheral resources. VPSS is a hardware module in DM6446 that is specifically responsible for video input and output. It consists of a video processing front end (VPFE) and a video processing back end (VPBE). The system analog video signal is decoded into YUV422 format video data by the TVP5150 decoder and then transmitted to the CCD controller of VPFE. Then, the data temporarily stored in the VPSS internal buffer is transmitted to the external DDR SDRAM through the EMIF interface. The screen display (OSD) module of VPBE submits the video data in the form of YCbCr to the video encoder (VENC), which is then output to the VGA or CVBS interface through the video encoder. The hardware topology of the processing system in this article is shown in Figure 1.

Figure 1 DM6446 system platform hardware structure diagram

2 System Software Design

Figure 2 shows the structure of the system hardware platform software processing module. As shown in Figure 2, the software part mainly completes three tasks: image acquisition and storage, ARM and DSP communication, and video image processing.

Figure 2 Block diagram of the DM6446 system platform software modules.

The ARM side is responsible for initializing the system and controlling the data acquisition and storage. When the system is initialized, VPSS is configured as field mode, and VPFE is responsible for continuously transferring the acquired video data to the memory buffer. Since the DSP side cannot read the image at the same time when the ARM side stores it in the memory, in order to ensure the correctness of data processing, this paper adopts a double buffer mechanism, that is, setting two consecutive frame buffers VIDEO BUF0 and video BUF1, and the video data is alternately refreshed to these two buffer areas.

The communication between ARM and DSP cores is realized through memory sharing and interrupts. ARM sends an interrupt signal to DSP through register ARM2DSP0, and DSP uses register DSP2ARM0 to send an interrupt signal to ARM. The party sending the interrupt fills the command in the shared memory before the interrupt signal is sent, and the party receiving the interrupt reads the command in the interrupt function. Other modules execute corresponding tasks according to the current command.

The video image processing is performed on the DSP core. The collected video image data is in YUV422 format. During the algorithm processing, only the Y component of the video data is read. Then, the video data is subjected to operations such as difference accumulation background modeling and motion area detection to detect moving targets in the video sequence. Finally, the DSP is responsible for storing the detected video data in a fixed display buffer area, which is read and displayed by the VPBE.

3. Video Moving Object Detection Algorithm Design

After obtaining the video data, it is necessary to further detect the moving target in the video (moving vehicles). For applications with fixed cameras, background difference technology is generally used to detect moving targets in the video, but the background difference method is constrained by the background modeling effect. At present, there are many methods for detecting moving targets in the video, such as those based on optical flow field, target model and differential image. The target detection method based on optical flow field is sensitive to noise and has a large amount of calculation, resulting in poor real-time performance of the algorithm. The detection method based on the target model generally requires the establishment of a three-dimensional model of the target, then projecting the model onto a two-dimensional plane, and then matching it in the image. The detection method based on differential image is more commonly used and can be divided into two methods: adjacent frame difference and background difference. The background difference method is a popular method in video moving target detection. This paper uses the background difference method to detect the target moving area. This paper targets embedded applications. Starting from the processing efficiency of the detection algorithm, the scene background model is obtained by using the background modeling method based on difference accumulation described in the literature [4]. The detection of moving targets in the video mainly goes through four steps: difference accumulation background modeling, motion area detection, morphological filtering and region growing method video moving target positioning.

3.1 Difference Accumulation Background Modeling

The difference accumulation background modeling method first establishes a difference image. Let the difference image be F(i, j, t), the current video frame be f(i, j, tc), and the reference frame be f(i, j, tb). Then, a difference image can be obtained by comparing the current frame with the reference frame, which can be expressed as formula (1):

Here Tf is a specific threshold that determines the degree of difference.

If the grayscale difference between the two images is greatly different in their coordinates, the value of F(i,j,t) is assigned to 1, otherwise it is 0. In this way, an image matrix with a pixel value of 0 or 1 and the same size as the input image can be generated, which is called a difference image. Now consider a set of m-frame video sequences {f(i,j,t1), f(i,j,t2), ..., f(i,j,tm)}. This paper selects the f(i,j,tc-3)th frame of the video as the reference frame. When the system is initialized, the first frame of video data read is assigned to the reference frame. The reference frame is continuously iterated with the input of the video. When the program is implemented, it can be iterated as follows:

The difference image describes the movement of objects in the video image. If there is a relatively stable non-moving area in the continuous image sequence, it can be considered as the background. The difference image matrix changes continuously with the video frame. According to the degree of change of each pixel value of the difference image, a difference accumulation dynamic matrix D (i, j, t) can be obtained. As shown in formula (2):

λ records the depth of each pixel difference accumulation. The number of cells and aspect ratio of the dynamic matrix are the same as those of the difference image matrix. When the value at a certain position of the difference image is 0, the counter at the same position of the dynamic matrix is ​​incremented by 1. There is one such counter at each cell position of the dynamic matrix. During the continuous processing of the video sequence, if the counter value of the dynamic matrix element is equal to λ, it is considered that the grayscale value of the pixel at the corresponding position has not changed within the specified range and can be used as a background pixel.

The background is updated using the weighted average method. Let the background before the update be Bold (i, j, t) and the new background be Bnew (i, j, t). Combined with the coefficients, formula (3) can be used to obtain the current updated background. The background of the application scene will be automatically updated as new video frames are input. Figure 3 illustrates the background update effect of the difference accumulation method.

Figure 3 Schematic diagram of the difference accumulation background modeling process

3.2 Motion Region Detection

After obtaining the background model, the background difference method can be used to obtain the background difference map. Assuming the background difference map is Bd (i, j, t), we have:

The moving vehicle area in the video can be obtained by binarizing the background difference map. The binarization threshold in this paper adopts the Otsu method (maximum inter-class variance method) in the literature [5]. The Otsu method divides the image into two parts, foreground and background, based on the grayscale characteristics of the image. The larger the inter-class variance between the foreground and the background, the greater the difference between the two parts of the image. When part of the foreground is mistakenly classified as the background or part of the background is mistakenly classified as the foreground, the difference between the two parts will become smaller. Therefore, the classification with the largest inter-class variance means the smallest probability of misclassification. The image grayscale value that maximizes the inter-class variance and minimizes the intra-class variance is selected as the optimal threshold.

3.3 Morphological filtering

It is not enough to just binarize the background difference image. Holes and small noises caused by misjudgment may appear in the binary image. It must be further processed by morphological filtering to connect related areas and remove noise. The specific operations are corrosion and expansion. The role of corrosion is to eliminate irrelevant details from the binary image, and expansion can bridge the cracks. The process of corrosion first and then expansion has the effect of eliminating small objects, separating objects at fine points, and smoothing larger boundaries without significantly changing their areas. The process of expansion first and then corrosion has the effect of filling small holes inside objects, connecting adjacent objects, and smoothing their boundaries without significantly changing their areas. In this paper, a structural element of 3×3 pixels is selected to perform corrosion and expansion operations on the binary image in succession.

3.4 Target positioning using region growing method

Positioning refers to obtaining the spatial position of a moving target in a video. This paper uses the region growing method to mark the position of the target. Region growing is an effective image segmentation algorithm that groups pixels or sub-regions with similar properties to form a region and continuously expands the region. The basic method is to start with a set of seed points and attach adjacent pixels with similar properties to each seed in the growing region, gradually forming a region, and stop growing when no pixels meet the conditions for joining a region. The automatic selection of seed points should meet 3 criteria:

① The seed and its neighborhood must have a high degree of similarity; ② At least one seed must be generated in the desired area; ③ Seeds in different areas cannot be connected.

After morphological filtering, the moving vehicle area in the binary image has good spatial connectivity. The purpose of using the region growing method for positioning is to determine the foreground target area and mark it. This paper uses a rectangular frame to mark the foreground area: find the maximum value max (x, y) and minimum value min (x, y) of the coordinates in the four-neighborhood connected area, and then draw white straight line segments in the image and connect them into a rectangular frame. The region growing method target positioning can be simply summarized into the following four steps [4]:

① Initialize the marking matrix C (x, y, tk) to the binary image to be marked (set the gray value of the seed point to 255); ② Scan the C (x, y, tk) image row by row according to the gray value of the seed point. If a pixel point that meets the conditions is found, check the neighboring pixels with it as the center. The coordinates of the neighboring pixels that meet the merging conditions are pushed into the stack and set as the background point; ③ Use the foreground point position coordinates of step ② to obtain the target rectangular frame position coordinates (xman, yman), (xmax, ymax), push the rectangular frame coordinates into the stack, and return to ②; ④ Take out the rectangular frame position coordinates one by one and draw the rectangular frame.

4 Experimental results and analysis

To further illustrate the actual detection effect of the DM6446 real-time video moving vehicle detection system, this paper conducted detection experiments on different traffic video sequences. Since it is inconvenient to conduct effect tests at the actual traffic scene, the experiment used a "Sina S008" DVD player to provide video input signals, and the DVD player played the specified traffic video sequence. The output of the DVD was connected to the composite video signal interface of the video board through an AV cable, and the processing results were displayed by the LCD connected to the VGA port. This paper shared three sets of video sequences: Highway, Video.long.mjpg and Hzhighway. Highway comes from http://cvrr.ucsd .

edu/aton/testbed/,video.long.mjpg is the test sequence provided by the CD-ROM "OpenCV Tutorial: Basics" written by Liu Ruizhen, and Hzhighway is a selfie video sequence.

Figure 4 (a) to (d) show the processing flow of video moving target detection on DM6446. Figure 5 (a), (b) and (c) show the results of moving vehicle detection for Highway, Video.long.mjpg and Hzhighway video test sequences respectively.

The background of Highway is relatively simple. All vehicles are in motion and their direction of travel is the same as that of the camera lens. The detection effect is good. Although there are a few conjoined problems, the basic positioning is accurate. As shown in Figure 5 (a), it can be seen that the white rectangular frame includes the vehicle and gradually shrinks as the vehicle moves away from the camera.

Video.long.mjpg The framing range is small, and the vehicle is moving fast, so sometimes the vehicle cannot be completely framed. In the selfie sequence Hzhighway, the vehicle positioning is accurate, but sometimes the rectangular frame cannot frame a single vehicle and the size is unstable, as shown in Figure 5 (c). After comprehensive analysis, the following conclusions are drawn: ① If the background update speed is not fast enough, it is easy to cause the background to not be updated to the road surface in time when the stopped vehicle becomes the background and then resumes the moving state. ② The color of some vehicles is similar to the road surface, and part of the vehicle body will be mistaken for the background during the update.

Figure 4 Schematic diagram of video moving vehicle detection process

Figure 5: Experimental test results of the DM6446 video motion vehicle detection system.

5 Conclusion

This paper presents a technical implementation scheme of a video motion vehicle detection system based on DM6446, and describes the idea of ​​building the system's software and hardware platform. The experimental results given in this paper illustrate the technical feasibility of this system and its good detection performance. Although this system has achieved good detection results in several video test sequences, it still lacks adaptability to sudden changes in ambient lighting and interference from swaying branches. The algorithm needs further consideration, and the program also needs to be further optimized for the DM6446 hardware platform, which is also the author's further research work.