Digital wireless real-time video communication and shooting system

This wireless system is a video communication system based on a high-speed RF chip, and has an instant shooting function; in the video (continuous) mode, the resolution of QQVGA is used to transmit video data, which is 160×120 (@13fps), which can basically achieve real-time video. In actual applications, users can watch the scenery near the (distant) transmitting end on the LCD of the receiving end. When you see the scenery of interest, press the button and wait for a while to get a 1.3M pixel image, which is convenient and practical. To simplify the design, the system only uses 8-bit color depth and RGB data format, and does not use CCD camera chips or FPGA chips for logic control, saving costs.

The following focuses on the wireless video communication system with TMS320VC5402 DSP as the control core, describes in detail the interface design of the two major parts: DSP and camera chip, and DSP and RF chip, analyzes the key points in the design, and finally gives some DSP assembly code.

1 Composition and working principle of wireless real-time video system

1.1 Introduction to OV9640 Camera Chip

OV9640 is a high-performance CMOS image sensor chip launched by OmniVision of the United States. It supports 1.3 million pixel image capture and multiple resolutions, including l280×960, VGA, QQVGA, CIF, QCIF, etc. and multiple data output formats, such as Raw RGB, YUV (4:2:2), YcbCr (4:2:2), etc.; supports 8-bit or 16-bit data output; it can be programmed through the SC-CB interface to realize various basic functions of image processing, such as exposure control, white balance, color saturation, gamma control, etc.; the chip has low voltage requirements and can be used in embedded mobile devices.

1.2 Introduction to nRF24L01 RF Chip

nRF24L01 is a single-chip wireless GFSK transceiver chip from Nordic, Norway. It operates in the 2.4-2.5 GHz ISM band, with a maximum wireless transmission rate of 2Mbps. It uses an SPI interface with the MCU for control and data transmission. Compared with its previous generation product nRF2401, nRF24LOl has better performance and lower power consumption. It can support up to 6 data channels, and each channel supports Enhanced ShockBurst (ESB) technology, with automatic response (AACK) and automatic retransmission (ART) functions, which reduces the burden on the MCU, reduces the packet loss rate of wireless data, and improves the efficiency of two-way transmission. When ESB is turned on, nRF24L0l will automatically switch to receive mode after sending the data packet to wait for the other party's response, and will implement automatic retransmission according to the register settings.

1.3 System Hardware Circuit

The hardware circuit consists of two parts: the sending (acquisition) end and the receiving (storage and display) end. The wireless link in the 2.4 GHz frequency band is realized through the high-speed RF chip. The sending end uses TMS320VC5402 DSP as the control core, OV9610 camera chip as the video (or image) acquisition front end, AT29LV1024 Flash ROM as the DSP boot program storage chip, K4S161622H IMB capacity SDRAM as the program running space and video data buffer, and the video data is finally transmitted through the RF chip nRF24LOl; the hardware structure of the receiving end is basically the same as that of the sending end, and the OV9640 at the front end is replaced with the LCD display at the back end. The overall structural block diagram of the whole system is shown in Figure 1.

1.4 System working principle and process

1.4.1 The sender works in video streaming mode

The transmitting end is controlled by DSP as the core chip. DSP is initialized after power-on, and the code in Flash ROM is loaded into SDRAM through BootLoader to realize high-speed operation of the system to speed up data processing, and HPI interface is set as general I/O. Then, nRF24L01 is set to the transmission mode through McBSPO buffered serial port, and the data packet containing the predetermined address is sent out to detect the receiving end. nRF24L01 will automatically switch to the mode of waiting for the response signal. If there is a correct receiving end (the address matches), nRF24L01 notifies DSP through INTO interrupt, so that DSP sets nRF24L01 to the transmission mode again, and immediately initializes OV9640, realizes SCCB bus through McBSPl buffered serial port, starts the camera and sets it to continuous frame mode. The resolution at this time is standard QQVGA, that is, 160×120 (@8bit). Finally, DSP converts the 8-bit parallel data obtained from D[7:O] into serial format, sends it to nRF24L01 through SDRAM buffer and McBSP0, and transmits the video data. If the correct receiving end is not detected (no INTO interrupt occurs), DSP will wait for INTO to occur or until the user turns off the power.

1.4.2 The transmitter works in shooting mode

During the video streaming process, nRF24LOl can monitor the air signal and automatically respond at the same time. If a photo taking notification is received from the receiving end (press the button), OV9640 is set to standard shooting mode with a resolution of l280×960 (@8bit). Then, DSP sets nRF24L01 to send mode and sends the frame data at this time. After the image data is sent and the confirmation signal is received, the system will return to the video streaming mode. If the reception is unsuccessful, the automatic retransmission function of nRF24LOl will ensure the integrity of the data transmission.

1.4.3 Workflow at the receiving end

The power-on initialization of the receiving end is basically the same as that of the transmitting end, but nRF2dL01 (according to the predetermined address) must be set to the receiving mode to receive the detection signal. After detecting the matching address, the automatic response function of nRF24L01 will send a response signal to the transmitting end to confirm the receipt of the signal. At this time, the "handshake" between the two parties is successful. Then, the DSP is notified through the INTO interrupt, so that the DSP sets nRF24LOl to the receiving mode again to receive the continuous video stream from the transmitting end, and turns on the LCD module to display the video. Finally, the DSP buffers the video stream through SDRAM and sends it to the LCD for display (if other back-end modules such as LCD are parallel interfaces, the data needs to be converted into parallel data format). At this point, the system can realize the wireless transmission of real-time video data, and the resolution of the real-time video stream is QQVGA160×120 (@13fps).

During the display of the video stream, if the user presses the photo button, an INT1 interrupt is generated to notify the DSP, and the DSP sets the nRF24LOl to the sending mode and sends a photo notification signal. After receiving the response signal, the nRP24L0l returns to the receiving mode to prepare to receive the image data. After the reception is completed, it will automatically send a confirmation signal to indicate that the image data has been successfully received. Finally, the DSP passes the image data to the back-end module for processing. So far, the system has realized the wireless shooting function of the image, and the photo effect is 1280×960 (@1.3 million pixels), which can basically meet the photo requirements.

The workflow of the video system receiving and sending ends is shown in Figure 2.

2 DSP interface design

2.1 Interface design between DSP and OV9640

The chip is controlled by the SCCB bus independently developed by OmniVision, using three-wire connection. SCCB_E is the serial port enable/disable signal line, SIO_C and SIO_D are the serial port clock line and data line respectively. The system only controls OV9640 without knowing its status. The McBSPO port of DSP is used for connection, where HD3 is used to open the serial port, BCLKX0 and HDX0 are used to send clock signals and control data respectively. The specific hardware connection is shown in Figure 3.

Worth noting:

① When using the DSP's HPI interface to connect OV9640, in order to obtain signals such as field frequency, line frequency and 8-bit video output, the DSP's HPIENA pin must be pulled down so that HD[7:0] can be configured as a general I/O interface to achieve start-stop or working mode control of OV9640.

② Set the output size to 1280×960 or QQVGA by modifying the COMC register of OV9640.

③Modify register COMH, set the output format to 8-bit RGB format, and set it to Master mode.

④FREX is the enable/disable signal of the continuous frame mode, but it is disabled by default. The HD4 pin of the DSP should be configured, its level should be pulled high, and then the EXPSTB pin should be set high with HD3 to achieve continuous frame data output; setting FREX low can return to the (single frame) shooting mode.

⑤ Calculate the data transmission rate. The chip outputs parallel 8-bit image signals and field frequency and line frequency synchronization signals. The size of a frame image (160×120) in the video is:

160×120×8=153600 bits/frame

If the maximum transmission speed of nRF24L01 is 2Mbps, it can send

2000000/153600≈13 frames

That is, the system can transmit video signals at a speed of 160×120 (@13 fps).

⑥Parallel-to-serial conversion of output data.

OV9640 outputs 8-bit parallel data, while nRF24L01 is a serial interface, so the parallel data needs to be serialized. The 8-bit parallel data format is HREF, Bll, G2l, B22, G12..., HREF, G21, R22, G23, R24... By programming the timing of the transmitter and the receiver, the PCLK, HREF and VSYNC signals obtained from HD[2:0] can be omitted and not included in the wireless transmission. Therefore, when serializing, the 8-bit data read from D[7:0] is placed in the SDRAM buffer in order from high to low, and then passed to nRF24LOl for transmission.

2.2 Interface design between DSP and nRF24L01

2.2.1 Design points

DSP uses McBSPl to connect directly to the chip. CSN is the SPI chip select pin, which is valid at low level. The XF pin of DSP is connected to the CSN pin; CE is the transceiver mode selection pin, which is controlled by HD7 for high and low levels. The connection of other pins is shown in Figure 3. Note the following when programming:

① Each time before sending a command to nRF24L01 via SPI, CSN must be jumped from high to low.

That is, after each instruction is executed, CSN must be set high to continue sending the next instruction.

②The SPI of nRF24L01 latches data on the falling edge, so McBSPl should be configured as "falling edge with delay".

③The IRQ pin is valid at low level, and each interrupt generated to the DSP must be cleared by writing "l".

④ If the transmitter needs to receive a response, data channel O should be configured to receive the response signal, and the receiving address (RX_ADDR_PO) should be consistent with the sending address (TX_ADDR).

⑤The chip must go through the Standby mode before entering the TX or RX mode, so when switching between TX and RX modes, the CE should be pulled low first to enter the Standby mode.

⑥ The instruction to write register can only be executed in Powerdown or Standby mode, so CE should also be pulled low before modifying the register value.

⑦ The following 2.2.2 and 2.2.3 do not enable the AACK and ART functions. This is because when the system is running in continuous video streaming mode, it only requires a high data transmission rate to meet real-time requirements, and does not require error correction and retransmission. However, in shooting mode, these two functions should be enabled to ensure the integrity of image data.

2.2.2 ESB sends data

① Set the configuration bit PRIM_RX to low;

② Keep CSN at a low level and send the address (TX_ADDR) and data (TX_PLD) of the receiving end;

③ Set CE high to start data transmission;

④After the data is sent, a TX_DS interrupt is generated;

⑤Set CE to low to enter Standby mode.

2.2.3 ESB receives data

① Set the configuration bit PRIM_RX high and CE high, then after 130μs, nRF24L01 starts to monitor the air signal;

②RX_DR generates an interrupt after receiving a legal data packet;

③RX_P_NO in the status register records the received data channel;

④Set CE low to enter Standby mode;

⑤MCU gets data through SPI.

2.2.4 Some program examples

(1) Write nRF24L01 register

(2) Read nRF24L01 registers

(4) Sending data

Assuming that the data width has been set to 32 bytes in (3), W_TX_PAYLOAD.macro tx_start_byte

(5) Receiving data

Assuming that the data width has been set to 32 bytes in (3), R_RX_PAYLOAD.macro rx_start_byte

Conclusion

The wireless communication system with DSP as the core adopts high-speed and low-power wireless RF chip nRF24L01, high-performance TMS320VC5402 digital signal processing chip and multi-function camera chip OV9640 in hardware, so that it can realize powerful video communication function, and optimize the design at the software level to shorten the waiting delay and reduce power consumption. This system has strong scalability. For example, adding a voice codec chip can realize synchronous transmission of wireless audio and video; adding a video compression chip can increase the frame rate; using two (or more) wireless transceiver chips at the same time can realize full-duplex and improve the air data transmission rate; using a high-resolution camera chip can obtain better quality video and image effects, etc., but it also increases the difficulty of hardware wiring and software design.

In short, wireless video and wireless image communication are the key development directions of the future wireless field. This design realizes real-time QQVGA video transmission at a certain frame rate, which has certain reference value for the design of similar systems in the future.