DSP video processing driver based on GIO/FVID

Introduction

With the development of the times, DSP technology has been widely used in video processing fields such as remote monitoring, video phones, and industrial inspections. For different video processing systems, different video equipment will be used, so it is necessary to design video equipment Drivers provide a unified interface for high-level applications to operate the underlying hardware. As long as high-level applications are developed in compliance with this driver interface standard, they can run on different hardware platforms with the same interface and have good versatility and portability. At the same time, as long as high-level application designers know how to use device drivers The API interface provided eliminates the need to understand the specific implementation of the underlying hardware, which can greatly improve the development efficiency of the overall video system.

For video devices, TI has proposed corresponding video device driver models, but these models are mainly designed for the 6000 series of high-end DSPs, or even video processing-specific DSPs such as DM64X, while low-end DSPs such as the TMS320F2812 (referred to as F2812) DSP The processor has small internal storage space and does not have a dedicated video interface like DM64X. To address this type of problem, this article proposes a method to simplify and transform the TI video driver model so that the video device driver takes up as few system resources as possible. To complete the operation of video hardware devices, this video driver model cutting method can be used as a reference for video processing systems using low-end processors.

1 Peripheral driver development model based on DSP/BIOS

In order to develop DSP peripheral drivers, TI launched the DSP/BIOS Device Drive kit ^[1] , which defined a standard device driver model and provided a series of API interfaces. , as shown in Figure 1, the peripheral driver is divided into two layers:

① Class driver (class driver), the class driver is used to provide an interface for the application program. This part of the program has nothing to do with the device. Its main functions include maintaining the device data buffer, providing upward API interfaces for application layer program calls, and coordinating the application program to the outside world. Assume the synchronization and blocking of operations, provide an adaptation layer downward to connect with the mini driver layer, and realize the mapping of API interface functions to mini drivers. The class driver has nothing to do with the hardware. As long as the peripheral driver model is selected, the class driver Once it's settled, it doesn't require much modification.

②Mini driver (mini driver), mini driver is related to the device, so designing mini driver is the focus of peripheral driver development. The interface format of mini driver and class driver layer is unified, but mini driver has no influence on the underlying hardware. The operation varies according to different hardware platforms. The mini driver receives the IOM_Packet command package sent by the class driver layer and decides what operations to perform on the underlying hardware.

Peripheral driver models can be divided into the following three categories ^[2] :

1) PIP/PIO model. Based on the I/O model of data pipelines, each pipeline maintains its own buffer. When data is written into the buffer or data is taken out from the buffer, the notifyReader and notifyWriter functions are triggered to achieve data synchronization.

2) SIO/DIO model, a data-based I/O model. A data flow is one-way, either input or output, and the SIO/DIO model uses asynchronous methods to operate I/O, for data reading and writing. , processing can be carried out simultaneously ^[3] .

3) GIO model, a general-purpose I/O model, is very flexible, has no adaptation layer, and directly operates mini-drivers. It is mainly used to design new device driver models.

2 Video processing system hardware platform

The hardware platform is shown in Figure 2. The system uses TI's F2812 DSP as the central processor, uses an analog camera to collect video signals, and then uses the SAA7111 video decoding chip to convert it into a digital video signal in BT601 format. , DSP processes the digital video signal, then writes it into the output frame buffer AL422, and controls the video encoding chip ADV7177 to convert it into an analog TV signal for output. The entire system uses one CPLD - IspMach LC4128 to coordinate the timing relationship between various chips.

3 Video device driver development

3.1 Selection of device driver model

As introduced above, commonly used driver models include 3 categories: PIO, SIO and GIO. Comparing these three models, we can know that PIO supports lower-level communication and is suitable for designing relatively simple peripheral drivers. For example, the audio collection and playback implemented on TI's 6X11DSK board are generally based on the PIO model ^[4] , and the SIO model has a good buffer allocation and recycling mechanism, which is more suitable for describing video devices. However, many functions of SIO cannot be used in this system, and the purpose of the GIO model design is to solve new devices for special hardware, so the final consideration Use GIO's device driver model. ^{The GIO model [3]}

originally designed by TI was actually flawed, mainly in the issue of data buffer management. After the application obtained the buffer for data processing, it was unable to return the buffer to the device driver, so TI When the DM642, a DSP chip mainly used for video processing, was launched, the GIO model was improved and a FVID model specifically targeted at video equipment was proposed ^[5] . The FVID model is built on the GIO model, with FVID_Alloc , FVID_exchange, and FVID_free functions encapsulate the GIO_submit function in the GIO model, solving the problem that the driver in the GIO model cannot recycle the buffer. In addition, the FVID model also specially designed the FVID_frame structure, which contains commonly used video signal information, such as the number of rows, columns, YUV structure, field frequency, etc., which is very suitable for describing video data frames, but FVID is mainly for DM64X System design, many functions of DM64X are not available on F2812 DSP. Therefore, this design simplifies the FVID model to a certain extent for the F2812 DSP radio frequency processing system, retains the class driver, and rewrites the mini driver layer program. 3.2 Video processing program running process Based on the designed video driver, a typical video processing application is developed. The running process is shown in Figure 3. First, use the FVID_create function to establish two video channels, GIO_capture and GIO_play, and then use GIO_capture The FVID_control function of the channel issues cmd_start and collects 1 frame of video data. The application uses the FVID_alloc function of the GIO_capture channel to apply for the collected data frame from the driver. After processing, the modified data frame is returned to the driver using the FVID_exchange function. Finally Then call the FVID_control function of the GIO_play channel to issue the cmd_display command to output the data frame. As can be seen from Figure 3, these FVID_xxx interface functions called by the application will automatically be mapped down from the class driver layer by layer to the mini driver layer program, and the mini The layer program can directly manipulate the underlying hardware device to complete the entire video collection, processing and display process.

3.3 Design of mini-driver

The mini-layer driver is the focus of the entire design. Its implementation method is introduced in detail below. The mini-layer driver mainly consists of several functions listed in Table 1.

The specific implementation of each function is as follows:

1) mdBinDev function. It is called when the application establishes a device interface (such as the FVID_create function) to complete the initialization of the external device, and its corresponding function is the mdUBindDev function. Using the mdUBindDev function will make the device in an invalid state and can no longer be used.

2) mdCreateChan function, use this function to establish a communication channel for the application and driver, and apply for a buffer for each channel. In the FVID model released by TI, 3 buffers are allocated for each channel, taking turns with External devices exchange data, and each buffer corresponds to 1 frame of video data. This design is completely feasible in a system like DM642 that can expand the capacity of SDRAM. However, for this system, the F2812 DSP only expands 512K×16 bits externally. SRAM not only serves as a frame buffer for video input, but also stores part of the program, so the storage space is not enough. Therefore, this design is simplified and uses a two-buffer rotation mechanism for the video input device, as shown in Figure 4(a) As shown in the figure, for the video output device, the AL422 FIFO is used as the hardware frame buffer without allocating a buffer for it in SRAM. Corresponding to mdCreateChan is the mdDeleteChan function, which is used to delete the device channel and release buffer resources.

3) mdSubmitChan function. Responsible for managing the buffer, accepting and processing the three commands FVID_ALLOC, FVID_EXCHANGE, and FVID_FREE issued by the application program. The FVID_ALLOC command corresponds to the process from (a) to (b) in Figure 4. The application program takes out the latest from the two buffers. A frame of video data is processed, and only one buffer is left to receive data input from an external device. FVID_EXCHANGE corresponds to the process from (b) to (c) in Figure 4. The application has processed one frame. data, return this frame of data to the driver, prepare it for display, and read in a new frame of data for processing. FVID_FREE corresponds to the process from (c) to (a) in Figure 4. The application program will process the processed data. The frame is returned to the driver without requesting a new data frame from the driver. The above three commands are for the video input interface GIO_capture, but for the output device interface GIO_play, there is no buffer allocated in SRAM, so its mdsubmitChan function is internally set to an empty function?br>
4) mdControlChan function, used to operate External video device, completes the collection and output of video data. The commands accepted by the mdControlChan function of the two device interfaces GIO_capture and GIO_play are different; the mdControlchan function of the

video input GIO_capture interface only accepts the cmd_start command to complete the processing of 1 frame of video data. Collection, while the mdControlchan function of the video output GIO_play interface only accepts the cmd_display command to complete the output of video signals.

3.4 General method of video driver model cutting

The GIO/FVID video device driver prototype designed by TI is relatively complex and takes up a lot of system resources. To make it applicable to more general low-end processor systems, it must be transformed and To reduce the cost, the following aspects should be paid attention to during the transformation:

1) Blocked I/O operations. TI's 6000 series DSP has EDMA function and does not require CPU intervention to transmit data, while DM64X also has a dedicated radio frequency interface to transmit data. The data will not occupy the external expansion bus, so the processing and input and output of video data can be parallelized. However, low-end processors do not have such a function. Video devices are generally connected through external expansion buses, so the processing of video data is The operation must be designed as a blocking I/O operation. The video data input/output process is completed by the CPU, and it must be ensured that the operation of the video device will not be interrupted by other operations.

2) Management of video data buffers. Although the 3-buffer model used in the GIO/FVID video device driver prototype has perfect functions, it takes up too much storage space, so the actual video processing system must be adjusted. , which is a two-buffer or even single-buffer model. If the object has an independent hardware cache for the output device, you can consider no longer allocating a dynamic buffer for it.

3) For the operation of video devices, the mdControlChan function is mainly used to operate external video devices. It is enough to retain the operations that are useful to the actual system, and many operations defined in the GIO/FVID video device driver prototype can be omitted.

4 Summary

This article introduces the peripheral driver model based on DSP/BIOS, and focuses on the specific hardware platform of the video processing system based on F2812 DSP, focusing on the process of developing GIO/FVID device drivers and targeting low-end processor systems. Video driver model cutting method, this video driver provides strong support for the development of various video processing applications (such as JPEG image compression, MPEG video compression, video monitoring programs, etc.). The development method of device driver introduced in this article , it has good reference for similar video processing systems, especially for video processing systems using TI2000 series DSP with relatively limited system resources.