A fingerprint recognition device based on CMOS image sensor and USB

　　CMOS image sensor is a new type of solid-state image sensor that has developed rapidly in recent years. It integrates the image sensing part and the control circuit in the same chip, which significantly reduces the volume and power consumption, and meets the requirements for highly miniaturized and low-power imaging systems. Compared with traditional CCD image sensors, CMOS image sensors also have many advantages such as high integration, simple control, and low price. Therefore, with the continuous advancement and improvement of CMOS integrated circuit technology, CMOS image sensors have been widely used in various general image acquisition systems. At the same time, as a high-speed communication interface between PCs and peripheral devices, USB has many outstanding advantages: easy connection, hot-swappable, no need to locate and run the installation program, no need to shut down and restart the system when connecting peripherals, realizing true plug-and-play; high transmission rate, USB 1.1 protocol supports 12Mb/s; does not occupy system hardware resources, can automatically detect and configure peripheral devices, and there is no hardware conflict problem.

　　Therefore, the fingerprint recognition device realized by using CMOS digital image sensor and USB interface data transmission has the advantages of simple structure, small size, portability, etc. This paper introduces the use of OMniVision's CMOS color digital image sensor OV762M and Cypress's EZ-USB AN2131QC USB control transmission chip (with enhanced 51 core integrated inside) to realize fingerprint information collection and USB transmission. At the same time, due to the mismatch between the output data rate of the fingerprint sensor (27MB/s) and the data transmission rate (12Mb/s) of the USB controller (AN2131QC), the system uses SRAM and CPLD to form an intermediate high-speed buffer.

　　system structure

　　The hardware block diagram of the fingerprint recognition system designed using AN2131QC, CPLD and OV762M is shown in Figure 1:

　　Figure 1. Schematic diagram of the fingerprint recognition hardware system

　　First, AN2131QC configures the window settings and other parameters of the fingerprint recognition sensor (OV7620) through I2C. After the optical lens forms an image on the image plane of OV762M, the CMOS image sensor (OV7620) performs spatial sampling and continuously outputs 8-bit digital image data Y[7∶M] at a certain frame rate (the frame synchronization signal of the output digital image data is VSYNC, the horizontal valid signal is HREF, and the output clock signal is PCLK). In order to match the output data of the fingerprint sensor with the data reading speed and timing of the USB controller (AN2131QC), SRAM (IS61C1024) and CPLD are used to form a high-speed buffer, and the fingerprint data collected by OV762M is cached using this high-speed buffer. Finally, AN2131QC realizes USB communication with the host computer and transmits the data in the high-speed buffer to the PC for corresponding image processing.

　　CMOS digital image sensor OV7620

　　The CMOS digital image sensor OV762M integrates a 664×492 photosensitive array, frame (row) control circuit, video timing generation circuit, analog signal processing circuit, A/D conversion circuit, digital signal output circuit and register I2C programming interface. After the photosensitive array obtains the original color image signal, the analog processing circuit completes the main signal processing tasks such as color separation and equalization, gain control, gamMA correction, white level adjustment, etc., and finally can output a variety of standard video signals as needed. The video timing generation circuit is used to generate multiple synchronization signals such as line synchronization, field synchronization, mixed video synchronization and multiple internal clock signals such as pixel clock. The external controller can set or read the working status, working mode and data output format of OV762M through the I2C bus interface.

　　AN2131QC sets the register of OV762M through the I2C bus interface to control the output frame rate to change between 0.5 frames/s and 3M frames/s, the output window is adjustable between 4×2 and 664×492 (the default output is the standard VGA format of 640×48M), and the black and white balance is set. According to the needs of fingerprint collection, the window output is set to: 320×288, and the output timing of OV762M after setting is shown in Figure 2:

　　Figure 2 0V762M output timing

　　VSYNC is the vertical field synchronization signal (also the synchronization signal for each frame, CMOS collects images by column), and its falling edge indicates the beginning of a frame of image. HREF provides an effective control method. When the output pixel rows and columns are respectively between the set windows, HREF is a valid high level, and valid video data is output at this time. PCLK is the output data synchronization signal, and a valid pixel Y[7∶M] is output on the rising edge.

　　Realization of High-Speed ​​Data Buffer Based on CPLD Technology

　　In the high-speed data buffer composed of CPLD and SRAM, CPLD acts as the controller of SRAM. Its internal circuit implementation block diagram is shown in Figure 3:

　　Figure 3 CPLD implementation of SRAM high-speed buffer controller

　　In Figure 3, ram_rd and raM_wr are the read and write signal lines output to the SRAM, raM_data and ram_addr are the data address buses of the SRAM; latch_f is the read and write enable signal of the SRAM, which allows write operations to the SRAM when it is at a high level, and allows read operations to the SRAM when it is at a low level; two 8-way three-state gates are used to isolate the bus. When writing to the SRAM, the output cpu_datA is in a high-impedance state, and when reading from the SRAM, the collected data signal Y [7∶M] is isolated; cpu_rds and vsync are the start read and write signals, and a single positive pulse sets the SRAM address to 0; cpu_rD is used as the SRAM fast read pulse, and pclk is the SRAM write pulse; irq is the write-full flag, which is used to provide an interrupt flag upward; the address generator is used to generate the SRAM address (IS61C1024 has 17 address lines).

　　Figure 4 Simulation waveform realized by CPLD

　　From the logic in Figure 3, we know that when writing to SRAM is allowed (latch_f=1) and the collected data is valid (href=1), the pclk pulse generates an address through the address generator (sync single positive pulse resets the SRAM address to 0), and writes the collected data Y[7∶M] into the SRAM. When it is full (writing a frame of 32M pixels × 288 pixels), the irq signal is valid, and latch_f is set low through an interrupt to allow the SRAM data to be read out (cpu_rds single positive pulse resets the SRAM address to 0). After that, cpu_rD generates an address through the address generator to read the data in the SRAM into the USB buffer. The above logic simulation waveform is shown in Figure 4 (due to the large number of data lines and address lines, only part of the signal timing is taken, cpu_datA is X, indicating that its value depends on the specific value on the SRAM data bus). From Figure 4, we can see that CPLD realizes the control of SRAM and forms a high-speed data buffer together with SRAM. [page]

　　Implementation of USB fast bulk transfer

　　USB control interface chip AN2131QC feature introduction

　　AN2131QC is designed based on USB 1.1 protocol, supports high-speed 12Mb/s transmission rate, embedded with enhanced 8051 microcontroller, 8kB RAM and a smart USB core transceiver, it contains an I2C bus controller and three 8-bit multi-function I/O ports, with 8-bit data bus and 16-bit address bus for external RAM expansion. Its structure is shown in Figure 5.

　　Figure 5 AN2131QC structure diagram

　　The USB differential transceiver inside AN2131QC is connected to the D+ and D- of the USB bus. The serial interface engine (SIE) encodes and decodes the serial data on the USB bus (i.e., implements the packaging and unpacking of the USB protocol), and performs error correction, bit filling and other signal standards required by USB. This mechanism greatly reduces the work of 8051 and simplifies firmware programming. The core microprocessor is an enhanced 8051 with an instruction cycle of 4 clock cycles and dual DPTR pointers. The instructions are compatible with the standard 8051. It uses internal RAM to store firmware programs and data. After power-on, the host downloads the firmware program and peripheral feature descriptors to the internal RAM through the USB bus (it can also be read directly from the on-board E2PROM), and then reconnects and re-enumerates according to the downloaded feature descriptors. This design can realize the implementation of software USB fast batch transfer.

　　After the collected fingerprint data is imported into the high-speed data buffer composed of SRAM and CPLD, to send the data to the host PC through the USB interface, AN2131QC must first read the data into the internal USB buffer. Therefore, the speed at which AN2131QC transfers data to the internal USB buffer will be the key to the speed of the entire USB data transmission. In order to make USB data transmission (reading data from the outside and transferring it to the PC) as fast as possible, many measures need to be adopted. The following discusses the USB batch transmission used in the design of the fingerprint recognition device firmware (AN2131QC program).

　　Normally, the AN2131QC core structure reads data from the outside into the USB endpoint buffer. The assembly program to be used is:

　　movx a,@dptr; read external data to acc register incdptr; add 1 to external address

　　incdps; switch DPTR pointer (the kernel has dual DPTR pointers, use dps to switch)

　　movx @dptr,a; put the contents of acc into the USB buffer

　　incdptr;USB buffer address plus 1

　　incdps; switch DPTR pointer

　　From the above program, we can know that after the data is operated in the register, there must be an "incdptr" and "incdps" instruction to complete the increase of the 16-bit address and the switching of the buffer pointer. In order to eliminate this internal consumption, a special hardware pointer provided by AN2131QC, namely the automatic pointer (only used for the internal buffer), is used. The 8051 loads the USB buffer address into two AUTOPTRH (high byte address) and AUTOPTRL (low byte address) registers, and the data written to AUTODATA is directly stored in the address buffer pointed to by AUTOPTR/H2L, and the kernel automatically increases the value of the 16-bit address in AUTOPTR/H2L. In this way, the USB buffer can write data sequentially like a FIFO, saving the "incdptr" instruction every time the internal USB buffer is written. At the same time, the kernel also provides a fast mode (only for external data operations). In this mode, when reading data from the outside "movx a, @dptr", the external data bus and the internal buffer are directly connected together. Since the fingerprint high-speed buffer composed of CPLD and SRAM has the nature of FIFO, the "incdptr" instruction is also saved when reading external fingerprint data using the fast mode. Combining the above two methods, the program to read external data into the internal buffer only needs one instruction: movx @dptr, A (dptR stores the address of the AUTODATA register). This instruction requires two 8051 machine cycles (8 24MHz clock cycles). In this way, a byte can be read into the USB endpoint buffer within 333ns.

　　On the data transmission side of the USB interface, when the PC wants to read data from a specific endpoint and sends an IN token, if the 8051 has not completed loading data to the USB endpoint buffer (reading external data) when an IN token arrives, the AN2131QC sends a NAK handshake signal to respond to the IN token, indicating that the PC should send another IN token later. In order to solve this waiting and achieve the fastest transmission speed, double buffering technology (endpoint pairing) can be used to enable the 8051 to load the next data packet of block data while the previous data packet is being transmitted on the USB bus.

　　in conclusion

　　The fingerprint device implemented by using CMOS digital image sensor OV762M and USB controller AN2131QC has a simple structure, small size and is easy to use. CPLD technology is used in the fingerprint recognition system to achieve high-speed buffering, solving the problem of speed timing matching; fast batch USB transmission technology is used to achieve fast data transmission, so that the transmission of fingerprint data reaches the highest speed (each frame transmission only uses 80Ms). The fingerprint data collected by the fingerprint device implemented by the method discussed above is reproduced by the PC as shown in Figure 6 (the left picture is the reproduction without any processing, and the right picture is the reproduction after smoothing, refinement and other algorithm processing).

　　Figure 6 The effect of collecting fingerprints and reproducing them (before and after processing)