The most powerful car surround view system design, achieving 360-degree safe driving without blind spots

With the rapid development of the national economy and the improvement of people's living standards, more and more families own cars as a means of transportation. How to park the car safely and conveniently has become a common problem faced by many drivers.

Traditional parking systems mainly use three methods to enable drivers to see the situation behind the car, namely rearview mirrors, reversing radar and reversing cameras. However, all three methods have a blind spot on the side of the car. For some complex roads, drivers can only see the front and back directions, while the sides of the car body are easily scratched by foreign objects on the roadside.

Therefore, the research and development of the 360° surround view system for automobiles has high prospects and applicability. This project uses Xilinx Spartan 6 FPGA for algorithm development and system control.

2. System Function Description

2.1 System Function

According to the design objectives of this project, the functions that need to be completed in this design are:

1. Collect image information from four cameras in front and behind the vehicle
2. The video information obtained by the four cameras is spliced ​​into a 360-degree panoramic image through video processing technology.
3. The 360-degree panoramic image must be coherent, without obvious signs of splicing

2.2 Time Performance

According to the design goals of this project, the car's surround view system should be able to process continuous video frame images in real time to ensure the safety of car driving.

3. Solution Design

3.1 System Working Principle

3.1.1 Theoretical analysis

In order to achieve the goal of 360° panoramic view, each camera must have a viewing angle of more than 90°, so we used a wide-angle fisheye lens with a viewing angle of 170 degrees in the design.

During use, because the lens viewing angle is large enough, there will be some overlap in the images from different cameras. In this way, as long as the camera positions are properly configured and the overlapping parts are properly spliced, a 360-degree surround image can be restored from the images of the four cameras.

3.1.2 Overall system structure

This system uses Xilinx Spartan 6 FPGA to develop system control and image processing algorithms. According to the analysis of system functional requirements and performance requirements, the system block diagram is as follows:

System Block Diagram

As can be seen from the figure, this system mainly consists of three parts, namely cameras (4), signal processing and display, etc. The video signal collected by the camera is sampled and sent to the signal processing part for image processing and splicing, and finally sent to the VGA LCD display for display.

3.2 System Solution Implementation

3.2.1 System Hardware Design

The system hardware design is shown in the figure below:

After the camera collects the image signal, it is sent to ADV7184 for PAL signal decoding. The decoded digital signal is sent to Spartan-6 FPGA for various image processing. After completion, the RGB signal is sent to ADV7123 for VGA format video output.

1. The ADV7184 is an integrated video decoder that automatically detects standard analog baseband television signals compatible with worldwide NTSC, PAL, and SECAM standards and converts them into 16-bit or 8-bit CCIR 601/CCIR 656-compatible 4:2:2 component video data.
2. Spartan-6 is the core device of this system, which has the following features:

• Designed for low-cost designs

• Very low static and dynamic power consumption

• Multi-voltage, multi-standard SelectIO™ interface bank

• High efficiency DSP48A1 Slice

• High performance arithmetic and signal processing

• Fast 18 x 18 multiplier and 48-bit accumulator

• Pipeline and cascading capabilities

• Pre-adder to assist filter applications

• Integrated memory controller module

• LUT designed for pipeline applications with dual flip-flops

• Block RAM with various granularities

• Low noise, highly flexible clock control

1. The ADV7123 is a high-speed digital-to-analog converter that has three high-speed, 10-bit, video digital-to-analog converters with complementary outputs, a standard TTL input interface, and a high-impedance, analog output current source for driving VGA outputs. It has the following features:
   • Throughput: 330 MSPS
   • Three-Channel, 10-Bit Digital-to-Analog Converter
   • Spurious Free Dynamic Range (SFDR)
   • RS-343A/RS-170 compatible output
   • Complementary output
   • DAC output current range: 2 mA to 26 mA
   • TTL compatible input

3.2.2 System software design

As shown in the figure, the work of this surround view system is divided into 8 steps, among which YCrCb to RGB format conversion, image denoising, shape correction, image cropping and stitching are all completed by FPGA.

1. After the ADV7184 completes decoding, it outputs a YCrCb signal, which is converted to RGB format for the convenience of subsequent processing. The conversion between YCrCb and RGB is as follows:
2. Since cameras (such as CCD, etc.) will introduce more or less noise when imaging, especially when the scene brightness is insufficient, the noise is obvious, which will affect subsequent processing. Therefore, it is necessary to denoise the converted signal.
3. Since a fisheye lens is used, the edge areas will be deformed, so shape correction is required.
4. After completing the above steps, the image can be cropped and stitched. There are many ways to stitch images. Here, you can first calculate the shape required for each lens, and then crop and stitch according to the calculation results.