Application of CMOS Image Channel in Ultrasonic Diagnostic Instrument

Abstract: This paper introduces a portable ultrasound diagnostic instrument control system that uses CMOS image channel for image transmission. The system is based on an ARM 32-bit microprocessor and an embedded Linux operating system. It completes the transmission and processing of diagnostic images in the CMOS image channel, and realizes real-time display and large-capacity non-volatile storage of diagnostic images. The construction of the diagnostic instrument is not only powerful and easy to expand, but also takes into account the needs of miniaturization.

1 Introduction

With the miniaturization and portability of medical diagnostic instruments, high-performance 32-bit microprocessors are increasingly being used in various small medical imaging devices. However, how to use these microprocessors to transmit and process real-time images is a problem that needs to be solved. Starting from the ARM 7 series, ARM processors have integrated CMOS (Complementary Metal Oxide Semiconductor) image sensor interfaces, which provides a possibility for solving the above problems. The processor can adapt to different CMOS image sensors through the CMOS image sensor interface. Through the CMOS interface, the image data will be sent to the system's CMOS image channel. The CMOS image channel is the connection path between the CMOS image sensor interface and the system AHB (Advanced High PerFormance Bus) bus, which consists of CSI (CMOS Sensor Interface) and PRP (Pre-Processor). This channel is specially used for high-speed image transmission, provides a variety of image format input and transmission methods, and can perform a variety of operations such as color space conversion and window adjustment in the channel. It is an ideal channel for the transmission of diagnostic images and other image applications.

This paper discusses a solution for a portable ultrasonic diagnostic instrument based on a 32-bit microprocessor ARM9E and embedded Linux. The system uses the CMOS image channel as the transmission channel for the diagnostic network image, sends the collected ultrasonic image information to the system bus, and realizes the real-time display of diagnostic images and large-capacity non-volatile storage of static images on the embedded Ijnux operating system platform. The implemented diagnostic system is not only powerful and easy to expand, but also takes into account the needs of miniaturization.

2 System Architecture

The structure of the portable ultrasound diagnostic instrument system designed in this paper is shown in Figure 1. The user inputs control commands to the system through the input device to complete the acquisition, display, storage and other related processing of ultrasound images. When executing the scanning command, the system sends the image acquisition command to the ultrasound image acquisition module with FPGA as the core through the serial port to control the ultrasound probe scanning drive circuit and the signal transmission and reception circuit to act synchronously. The received ultrasound signal is amplified and analog-to-digital converted in the ultrasound image acquisition module and sent to the FPGA. After completing the digital scan conversion (DCS) and interpolation operation, the two-dimensional B-ultrasound image information is obtained. After the image information is converted into a format, it is sent to the control system through the ARM chip CMOS interface 13. After being transmitted through the image channel, it is displayed in real time on the LCD (Liquid Crystal Display). At the same time, the pluggable large-capacity SD (Secure Digital Memory Card) card can save the required image. The saved image information can be processed offline, such as printing, movie playback, remote browsing, etc.

3 System Hardware Design

3.1 Hardware Block Diagram

This design uses Freescale i.MX21 with ARM926EJ-S as the core as the processor. The hardware framework of the control system is shown in Figure 2. The system is based on the ARM processor with compatible hardware peripherals. The ultrasound acquisition module completes the acquisition and transmission of images. The LCD displays the transmitted image information in real time, and a large amount of image data is saved through the SD card for later processing. Through the external USB HUB, the mouse, keyboard, and printer can be connected to the system. The locally stored diagnostic image data can be sent to the network port through the fast Ethernet control processor DM9000, and remote browsing can be achieved through Ethernet.

3.2 CMOS Image Channel

The block diagram of the CMOS image channel of this control system is shown in Figure 3. The input of ultrasound image data is realized by using the CMOS image sensor interface of the i.MX21 chip. The image data types supported by this interface include RGB565, RGB888, YUV422, YUV444, etc. After different types of image data are sampled by CSI 8 bits at the CMOS port, they are packaged into 32-bit image data and sent to CSI RxFIFO. Depending on the type of image data in CSI RxFIFO, the hardware and software implementation methods for sending and storing are also different.

[page]

(1) If the type is RGB565 and the image window does not need to be adjusted (display device is single), the image data can be directly transferred to the display buffer through the DMA channel, as shown in channel ① in Figure 3;

(2) If the type is RGB565 and the image window needs to be adjusted (display device diversification), the image data needs to go through the preprocessor (PRP1, where the image size is adjusted and sent to the display buffer, as shown in channel ② in Figure 3;

(3) If the type is YUV422, the image data needs to go through PRP, where the image size is adjusted and converted into a suitable color space before being sent to the display buffer, as shown in channel ② in Figure 3;

(4) If the type is YUV444 or RGB888, the image data needs to be color processed and converted through software programming and then sent to the PRP adjustment window and finally to the display buffer, as shown in channel ③ in Figure 3.

In order to make the image output by the LCD interface compatible with display screens of sizes such as QVGA, VGA, SVGA, and XGA, and to take into account the image transmission rate and image display quality, we use the RGB565 format and select the 3-channel②. Therefore, the image information collected by the ultrasound image acquisition module needs to be constructed in RGB565 format, so the timing diagram of the CMOS interface input image information is shown in Figure 4.

In the figure, Vsync is the field synchronization signal, Hsync is the horizontal synchronization signal, Pixelk is the pixel data input synchronization signal, and D [7:0] is the pixel data signal. When Vsync and Hsync are low, the input of valid image data can begin. The arrival of the rising edge of Pixelk indicates that data input is in progress, and one cycle of Pixelk will complete the input of one image data byte. In the process of transmitting a frame of image, that is, in one cycle of Vsync, Hsync will appear high for H times (H is the number of rows). In one cycle of Hsync, Pixelk will appear high for W*Pixelbytes times (W is the number of columns, and Pixelbytes is the number of bytes occupied by each pixel). The arrival of the next rising edge of the Vsync signal indicates the end of the image input process with a resolution of W*H.

After the image data in the CSI RxFIFO is sent to the PRP for window adjustment, it enters the memory through the AHB (Advanced High peRFormance Bus). The adjustment of the PRP window is achieved by setting the PRP control register.

3.3 Image display and storage

FrameBuffer is an interface provided by Linux for display devices. It is a device after the video memory is abstracted. Reading/writing this device can directly operate the video memory. Through the driver, a device file fb0 can be created in the file system for FrameBuffer. To complete the image input to FrameBuffer, the user only needs to complete the writing operation of the fb0 image. When the Pixel FIFO in the LCDC (LCD Controller) is empty or partially empty, LCDC requires to take the image data to be displayed from the FrameBuffer based on the burst transfer mode. When the transmission request of LCDC is received and confirmed by the bus arbiter in the storage controller, the image data in the FrameBuffer will be sent to the Pixel FIFO through a dedicated DMA channel, and then directly sent to the LCD display interface through the LCDC logic interface after byte flipping and background and foreground image synthesis.

The SD card can be connected to the control system through the MMC/SD interface, and the SD card can be mounted in the file system using the driver. The user can write an application to save the image data in the memory to the SD card, or read the image data in the SD card into the memory.

4 System Software Design

This control system is developed and designed based on the above hardware platform and embedded Linux operating system. First, the embedded Linux operating system kernel and file system are transplanted on the hardware platform, and the underlying driver of the peripheral hardware is realized. The application adopts modular design, in which the functional module design mainly includes GUI (Graphical User Interface) and WebServer program design, and the application module mainly includes acquisition module subroutine, LCD display module subroutine, printing module subroutine, SD card storage subroutine, Ethernet transmission subroutine, etc. This article focuses on the discussion of the image channel related programs.

As shown in Figure 3, the image data in the CSI RxFIFO is sent to the PRP through the dedicated bus channel ②, and the DMA channel ① is turned off by the internal logic controller. Therefore, in the CSI and PRP drivers, the initialization code of CSI and PRP should be modified as follows:

CSI initialization:

* (uint32_t *)GPIOB_GIUS &= ~0x3FFC00; // Turn off DMA channel

*(uint32_t *) CSI_CSICR1 |= 0x2; //Image data rising edge trigger

*(uint32_t *) CSI_CSICR1 |= 0x80; //Big endian format

*(uint32_t *) CSI_CSICR1 |= 0x10000000; //PRP enable

PRP Initialization:

*(uint32_t *)CRM_PCCR0 |= 0x8008000; //PRP clock enable

*(uint32_t *)EMMA_PRP_CNTL |= 0x10000; //PRP reset

*(uint32_t *)EMMA_PRP_INTRCTRL = 0x00000000; //Disable interrupt

*(uint32_t *)EMMA_PRP_SPIX_FMT = 0x2CA00565; //RGB565 format data input

*(uint32_t *)EMMA_PRP_SFRM_SIZE = 0x028001E0; //The input image window is 640 x 480

*(uint32_t *)EMMA_PRP_DISIZE_CH1=0x028001E0; //The output image window is 640 x 480

*(uint32_t *)EMMA_PRP_DPIX_FMT = 0x2CA00565; //RGB565 format data output

*(uint32_t *)EMMA_PRP_DLST_CH1 = 0x00000280; //The line step is 640

This design initially installs an LCD with a resolution of 640 x 480. After initialization as above, the image data will enter the PRP for processing from the CSI RxFIFO. In order to allow the upper-layer user to directly use the CSI device file descriptor without changing the device file, the following code is used to map the virtual address of the CSI device file to the physical address of the PRP output, so that the user can directly read the image data processed by the PRP from the CSI device file.

if ((csi_data_buf = (U32 *)__get_free_pages(GFP_KERNEL, 8))) {

prp_buf_phy_addr = virt_to_phys((void *) csi_data_buf);

printk("Buffer start: 0x%08x, PRP addr: 0x%08x\\n", (int) csi_data_buf, (int)prp_buf_phy_addr);

} else {

printk ("ERROR: cannot allocate buffer memory for driver ! \\n");

return -1; }

[page]

By modifying the driver as above, the user can write an application to display and store the image data after PRP processing. The program overview of the application is shown in Figure 5:

5 Test Results

The control system designed in this paper has been tested and the frame rate of the real-time display of diagnostic images can reach 20‰, which is greater than the frame rate requirement of 10 frames/s for portable B-ultrasound images. The LCD observation image is clear. The standard bitmap in RGB565 format is used to save the diagnostic image information. The data size of each picture is:

640 * 480 * (5+6+5)／8=614400 bytes

Using a 1G SD card as an external storage device, 1747 diagnostic images can be stored, which fully meets the requirements of portable diagnostic equipment and the needs of actual diagnostic situations.

6 Conclusion

This design uses the Freescale i.MX21 chip with ARM9E as the core as the central processor of the portable ultrasound diagnostic instrument. It uses its built-in CMOS image sensor channel to display the image information input by the ultrasound image acquisition module in real time and save the displayed image as needed. After testing, the control system works stably and reliably, meets the conventional diagnostic needs, and has high practical value.

The author's innovation: This paper adopts a dedicated high-speed CMOS image transmission channel to input the ultrasound diagnostic image into the control system and process it. On the basis that both the software and hardware meet the requirements, a detailed design implementation is given, saying that the designed system has achieved good results in the display frame rate and display clarity of the diagnostic image. It lays a good foundation for the design, upgrade and improvement of similar systems.

After the project is industrialized and put on the market, the annual economic benefits can reach 10 million yuan.

References:

[1] Zheng Zheng, Ji Jianjun, Li Sui, et al. A new type of ophthalmic ultrasound biomicroscope[J]. Journal of University of Shanghai for Science and Technology,
2005, (06). 512-516

[2] Yang Liu, Zheng Zheng, Huang Yunkai. Portable ultrasonic diagnostic instrument control system based on embedded Linux [J]. Journal of Shanghai University of Engineering Science
, 2007, 12(21). 361-364

[3] Hong Yuan. Embedded systems and their applications in ARM [J]. Microcomputer Information, 2007, 11(2). P27-29