Design of network camera based on CMOS image sensor OV7720

Network cameras have networking functions and maintain the functions of analog cameras to the maximum extent. They are undoubtedly the new stars in the field of monitoring. Network cameras generally use finished CCD cameras as the video capture front end. CCD cameras account for a large part of the hardware cost, and CCD cameras output analog signals, so video A/D converters must be added to the system. The development of CMOS cameras for network cameras is of great significance to reducing design costs and complexity.

1 Introduction to the chips used in the image acquisition system

1.1 Image sensor OV7720

OV7720 is a highly integrated CMOS Camera Chip sensor that provides all the functions of a VGA image processor on a single chip. A unique feature of OV7720 is that it has a large main light angle, which can significantly reduce the module height, which is a key factor in enabling the camera to be installed in current ultra-thin notebook computers; the second highlight of OV7720 is that it can work in 60f/s VGA mode or 120f/s QVGA mode to achieve the best performance of the device. OV7720 integrates perfect control of the image sensor, including exposure control, gamma correction, white balance, color saturation and hue control. All image processing functions can be programmed through its serial camera control bus (SCCB) interface to program the internal control register group. OV7720 uses proprietary sensor technology to improve image quality and produces clear, pure and completely stable color images by reducing or eliminating undesirable factors such as fixed pattern noise, smearing and floating. Its main technical parameters are shown in Table 1.

1.2 Camera control chip OV529

OV529 is a camera control chip that can convert signals from image sensors (such as OV7720) into data in compressed image format. Figure 1 shows the logical relationship between OV529 and the sensor, host and E2PROM. The technical features of this chip are: it can be used as a low-cost, low-power, high-resolution single-chip camera controller in mobile PDA systems and notebook computers; no additional DRAM is required; it has an 8-bit parallel camera chip data interface; built-in sampling circuit, clamp circuit and window circuit; it can be used for VGA, CIF, SIF, QCIF, 160×128, 160×120, 128×128, 80×64 and more image resolutions; built-in color conversion circuit can be used for 4 b/8 b grayscale, 16 b YUV, 12 b/16 b/24b RGB image preview; it has a serial camera control bus, a built-in microcontroller, an optional serial E2PROM memory, general I/O pins and 2 host interfaces (RS 232, 5wire), etc.

2 Overall system design

2.1 System composition

The overall design of the system is divided into two parts: hardware design and software design. The hardware design module mainly includes three parts: microcontroller module, network interface module and camera control module. The microcontroller module is mainly composed of the main control chip, DATAFLASH and SDRAM. The main control chip is the core of the whole system and is responsible for the scheduling of the whole system. The embedded Linux kernel and its file system, application software and system configuration files are solidified in DATAFASH. SDRAM is used as memory for system operation. The network interface module is mainly used to cooperate with the main control chip to transmit MPEG-4 code stream. The camera control module is mainly composed of OV529 and OV7720 chips, which transmit the video stream of the sensor to OV529 through SCCB (Serial Camera Control Bus); OV529 converts the video stream into MPEG-4 code stream by controlling OV7720, and completes the function of forwarding control camera commands. When the system starts, the microcontroller transfers the Linux kernel to SDRAM through SPI, and the system starts from SDRAM.

The software design part mainly includes three parts: embedded Linux transplantation, CGI control program and MEPG-4 decoding program. The embedded Linux system is stored in DAT-AFLASH controlled by AT91SAM7X256, which is responsible for the scheduling of the entire system software. The CGI control program is mainly responsible for the control of the camera and the related settings of the MPEG-4 video stream. The MEPG-4 decoding program is mainly responsible for decoding the MPEG-4 data stream obtained through the network.

2.2 SCCB Communication Specifications

SCCB (Serial Camera Control Bus) is a master-slave mode serial communication protocol defined by OminiVision, which is used to control the internal registers of the company's image sensor series chips. The protocol has a communication enable signal SIO_E, a clock signal SIO_C, and a data signal SIO_D. The SIO_E pin is no longer provided in the simplified pin package, and the protocol has become a two-wire serial communication. The highest serial communication rate supported by the protocol is 400 Kb/s, and it stipulates that the image sensor can only work as a SLAVE in communication. A SCCBMASTER can control 1 to 8 SLAVEs to work, which are identified by different IDs. When there is only one SLAVE, the fixed SLAVE ID is 42H (write) and 43H (read). The protocol conforms to the timing subset of the I2C port of the network camera CPU. In the design, OV529 is used as the MASTER of SCCB, and only one SLAVE OV7720 is included. The protocol requires that the data line must be stable during the high level of the clock signal and updated at the low level. The protocol supports three transmission modes: 3-byte write, 2-byte write, and 2-byte read. When transmitting data, each byte includes 9 serial data bits. In addition to 8 bits of data, there is also a 9th irrelevant bit or NA bit, which is used in systems with multiple SLAVEs to indicate whether the communication is complete. SLAVE can respond to this bit in two ways. One is that the SLAVE being written drives this bit to 0 after receiving the data to indicate success; the other is that SLAVE does not respond to this bit. In this case, the 9th bit of all SLAVEs remains floating. In the 3-byte write cycle, the MASTER must provide the sub-address, that is, the address of the register in the chip, and the 3rd byte is the written data. The read cycle does not provide a sub-address, but the MASTER must drive the NA bit (9th bit) after the data to a high level. Therefore, there must be a 3-byte or 2-byte write operation before each read cycle so that the SLAVE obtains the register address. The 2-byte write cycle does not write data. Its purpose is to provide the SLAVE sub-address before the read operation.

3 Hardware System Implementation

3.1 Microcontroller Module

The microcontroller module is the core component of the system, and uses Atmel's ARM7 chip AT91SAM7X256 as the main control chip of the system. It is an Atmel 32-bit ARM RISC processor with a small pin count FLASH microprocessor, with a main frequency of 55 MHz, fast processing speed, powerful functions, high cost performance, and can well meet the needs of embedded Linux systems. Its main role in the system is to configure the function registers of other chips when the system is powered on, schedule the entire system to work under normal working conditions, control the physical layer chip to send code streams through the on-chip Ethernet controller, and realize the control of peripheral DATAFLASH and SDRAM through the on-chip SPI bus.

3.2 Network Module

The on-chip Ethernet card port of AT91SAM7X256 communicates with the MII interface of the network physical layer chip DM9161E. When sending data, first set the transmission enable signal ETXEN to be valid. The data transmission end ETX0~ETX1 is connected to the ETX0~ETX1 pins of DM9161E as the data transmission channel, and the data is sent with the clock signal REF_CLK of DM9161E. The data receiving end ERX0~ERX1 is connected to the RXD0~RXD1 pins of DM9161E as the data receiving channel. The management clock signal EMDC and the management data input/output signal EMDIO are used to write and read chip control parameters. The MDINTR end of DM9161E is used to generate interrupt signals. The on-chip Ethernet card sends data through the DMA channel without affecting the normal operation of AT91SAM7X256. First, correctly set the parameters such as the number of bytes of the transmitted data block and the first address of the data block storage in the transmission control register and the transmission address register, and then read the data from the specified data storage area address in turn and send it to the internal transmission buffer. The MAC encapsulates and sends the data, and records the number of bytes transmitted at the same time until the data block is sent. When a group of data is sent, a DMA interrupt request is issued, and the AT91SAM-7X256 performs corresponding processing. The entire network subsystem circuit is controlled and scheduled by the AT91SAM7X256.

3.3 Camera Control Module

The camera control module includes the image acquisition sensor OV7720 and the camera control chip OV529. The OV7720 contains A/D conversion and DSP, which can directly digitize the image signal and directly output parallel digitized image signals to the OV529 through the camera interface.

The built-in MCU of OV529 performs various controls. It exchanges control information with OV7720 through the serial camera control bus (SCCB) to realize the control of image sensors and data acquisition. OV529 contains a compression engine to compress the image information frames collected from OV7720. The compressed signal is transmitted to the main control chip AT91SAM7X512 through the serial communication of OV529, and then processed by the MAC layer on AT91SAM7X512, and the video stream is sent to the network through the network interface.

4 Software Design

The embedded Linux system consists of four parts: the bootloader file, the Linux kernel file kernel, the virtual disk file ramdisk and the user space file use-r. They are placed in four partition modules in DATAFLASH. Different file systems are used according to the specific functions of different modules: bootloader, kernel, ramdisk. After the transplantation is completed, there is no need for dynamic changes, and the space-saving ROMFS read-only file system is used. Some configuration files that can be dynamically updated are placed in the user module. Many read/write operations are required, so the JFFS2 file system that supports dynamic erasure and storage is used.

4.1 CGI Control Program Design

The Common Gateway Interface (CGI) provides a channel for the Web Server to execute external programs. The CGI program is compiled into an executable file and placed on the server side for running. The server calls the corresponding CGI program according to the user's request, and encapsulates the dynamic information obtained by the CGI program into the page and sends it to the user's browser. The working process of the CGI of this system is to enter the IP address of the network camera in the IE browser on the PC side. The embedded network server feeds back the corresponding control page to the IE browser according to the requirements. The user fills in the form and submits it. The CGI program extracts the information and performs corresponding processing based on the information obtained, such as adjusting the camera's steering and focal length. The design of the CGI module mainly includes three parts: the configuration of the embedded Web server, the writing of HTML pages, and the design of CGI programs. The embedded Web server uses Apache, and its configuration is provided in the form of a configuration file and placed in the Ramdisk. Its configuration mainly involves configuring the root file path, configuring the path where the CGI external program is placed, and configuring the environment variables. The web page is written in a combination of HTML and SHTML. HTML has a faster parsing speed, and SHTML can embed external CGI programs in ordinary web pages. In this way, the default configuration of the system is fed back to the client side. The CGI program uses a combination of C language and Shell script, and the specific writing process is the same as programming under Linux.

4.2 Decoding Procedure

This part has been successfully made into a Toolbox plug-in. Once the plug-in is registered and installed on the PC, it can be easily called by the web page, so that real-time video viewing can be achieved on the PC. The actual picture after installation is shown in Figure 2.

5 Conclusion

This paper designs an embedded network camera using CMOS image sensor. After testing, the system is stable, reliable, and has good real-time performance. The image clarity is significantly better than products of the same price. It can fully meet the current market requirements for mid- and low-end network camera products and has a good market prospect.