Design of automatic focusing and image acquisition module based on PXA255

fighting

Design of automatic focusing and image acquisition module based on PXA255 [Copy link]

Abstract : This paper introduces an autofocus image acquisition system based on Intel's PXA255 embedded processor. It introduces the implementation of autofocus in FPGA and image acquisition in embedded Linux system in detail, and gives the hardware structure diagram of the system.
Keywords : autofocus; image acquisition; V4L video interface; embedded QT; discrete cosine transform

Introduction
The popular camera phones on the market generally do not have the autofocus function, which has little impact on camera phones with sensor resolution below 300,000 pixels. However, with the emergence of megapixel mobile phones and the adoption of mobile phone video recording functions, people are paying more and more attention to the autofocus function of cameras. Based on the Xhyper255 development board, this paper transplants QT/embedded as an embedded GUI on the development board in combination with the needs of project development, and designs the autofocus and image acquisition submodules. The autofocus part is implemented on the FPGA on the development board through Verilog HDL, and the image acquisition part is implemented using the video 4 Linux interface provided by the Linux kernel. The final acquisition program interface is designed using the embedded QT design tool designer.

Figure 1 Hardware platform structure diagram

Figure 2 DCT transformation module

Figure 3 DCT transformation flow chart

Figure 4 Autofocus Flowchart

Autofocus System Algorithm Based on Image Processing
Compared with traditional autofocus algorithms, the implementation of autofocus algorithms based on image processing does not require additional signal sources and corresponding receiving sensors, which is conducive to reducing the size of the device to reduce costs and reduce the power consumption of the device.

In this type of algorithm, the analysis and processing module directly processes the obtained video image to obtain the corresponding decision function, and the drive control module drives the stepper motor according to the obtained focus decision function information to drive the lens to move back and forth until a clearly focused image is obtained. Therefore, constructing a reasonable decision function has become the key to the autofocus algorithm based on image processing. The ideal focus decision function should be unimodal, unbiased, and able to reflect the polarity of defocus, and should also have strong anti-interference ability.

After making a corresponding comparison with the current autofocus decision function, combined with the characteristics of this system, the 2D-DCT transform is selected to remove the low-frequency components, and the remaining parts are added as the decision function. The formula of 2D-DCT transform is shown in Equation 1.

(1)
Here C(0)=1/, C(u)=C(v) (u,v≠0)

The development platform for the realization of the autofocus system is shown in Figure 1.

Using the Xhyper255 embedded development board, the image acquisition subsystem is built with the following hardware: 300,000-pixel CMOS image sensor OV7620, MCS51 microcontroller, USB controller OV511+, stepper motor and zoom lens.

The main features of OV7620 are: single-chip digital color image sensor; 1/3 optical format; digital video output format: 1~500 times automatic exposure range; automatic gain and automatic white balance; can perform brightness, contrast, saturation, gamma correction and other adjustment functions. The 664×492 image array scans the original R, G, B color image signals, which can be converted into YUV and other signal output forms according to output requirements after exposure, correction, white balance adjustment and other processing by the analog processing circuit. OV511+ is a dedicated USB interface control chip designed for CMOS image sensors.

FPGA implementation of DCT transformation
2D-DCT transformation is a common transformation in video compression. During the compression process, an image is divided into many 8×8 small blocks for transformation. The 8×8 2D-DCT transform is shown in formula (2):

(2)
Here C(0)=1/, C(u)=C(v)=1 (when u,v≠0).

After the transformation, the DC component is removed and the remaining components are added as the focus decision function. The maximum value of the function is the focus position.

1. Block preparation: Given a color image matrix of size 640×480, divide it into three matrices, namely the brightness matrix (Y), each of which is 8×8, with a total of 4800; the remaining in-phase matrix (I) and orthogonal matrix (Q) are differentiated into two groups of 1200 matrices, each of which is 8×8. Use the following matrices to map the RGB components to the Y, I, Q components:
Y=0.30R+0.59G+0.11B
I=0.60R-0.28G-0.32B
Q=0.21R-0.52G+0.31B

Calculate DCT for each 8×8 matrix Y, I, Q component. To calculate 2D-DCT, first perform 1D-DCT on each row of the matrix, and then perform 1D-DCT on the result matrix by column.

2. DCT module design: The DCT coefficients are implemented using a lookup table structure using a case statement. The program source code is not detailed here.

The implementation process of DCT transformation is as follows: the serial data is first placed in the input buffer (constructed using a ring register), then the product and sum operations are performed, and finally the data is output in the form of parallel data. These operations must be completed under the control module to ensure the correct timing.

The block diagram of the DCT transformation module is shown in Figure 2. The 8×8DCT transformation is actually a parallel operation of 64 pixels. The operation performed on each pixel is: the input data is multiplied by the DCT coefficient, and then the final result is obtained by adding them. The process is shown in Figure 3.

It can be seen from the formula that 8×8=64 times of calculation are required. After each calculation, i, j, u, v are changed accordingly, and the corresponding coefficients are found in the coefficient table again, multiplied with the new data, and the next operation is performed.

The implementation of automatic focusing
removes the low-frequency components in each 8×8 matrix. Then the corresponding high-frequency components are added, and the sum is used as the automatic focusing decision function and transmitted to the CPU. Programming realizes one PWM signal output to control the step size and direction of the stepper motor. Automatic focusing is realized. The process of automatic focusing is shown in Figure 4.

Implementation of image acquisition
The writing of image acquisition program is based on the Video4Linux interface provided in the Linux kernel. Video4Linux is an interface standard provided by the Linux kernel after version 2.2.0 for software development of network cameras, video capture cards, TV cards and other devices. This standard provides an API for the kernel, driver and application to communicate. The latest Video4Linux version is V4L2.

Use dual URBs to communicate in turn
For image acquisition applications that are time-sensitive but not demanding on data correctness, the USB bus defines the ISOC transmission mode, and USB cameras should use this transmission mode. In order to get image data as quickly as possible, the USB_ISO_ASAP flag should be specified in the URB.
urb->transfer_flags = USB_ISO_ASAP;
//Send this URB as quickly as possible

Any USB transmission in the Linux system is implemented through URB. To increase the speed, you can consider expanding the URB buffer or establishing two URBs. While waiting for one URB to be recycled, that is, when the image is being collected by the sensor, process and initialize another URB, and send it immediately after recycling. The two URBs are used alternately, which greatly reduces the extra time.

Use memory mapping and double frame buffering to improve efficiency.
　　The Linux system implements hardware operations through read, write, etc. They copy each other in the kernel and user memory spaces through functions such as copy_to_user() and copy_from_user(). However, for applications such as video acquisition that require a large amount of high-speed data transmission, this method consumes too much hardware resources. This problem can be effectively solved through the memory mapping method. First, use vmalloc() to apply for a sufficiently large kernel memory as an image data buffer space, where the image data brought back by the two URBs is temporarily stored; then use the remap_page_range() function to map it page by page to the user space. The user-state image acquisition processing program uses the mmap() function to directly read and write the kernel image buffer memory, greatly reducing the additional overhead. In addition, in order to further improve the frame rate, this paper adopts a double frame buffer method for image acquisition.

Conclusion:
　　　This system can be applied to the development of most intelligent products (such as the development of smart phones based on embedded Linux) by only making some modifications in the implementation of DCT algorithm and image acquisition interface according to specific hardware conditions. Therefore, it has good market application prospects.

rain

Strong everyone come and see