The application of 3D technology in automobiles helps achieve safe driving

    3D is not a new technology. It has been widely used in the field of consumer electronics, but it is still in its infancy in automotive applications, and there is still a need for breakthroughs in related technologies and solutions. Taking the application of safe driving as an example, during driving, it is difficult to eliminate blind spots in human vision. If the high-resolution camera images in the four directions of the car can be synthesized in 3D, so that the driver can freely change the perspective according to different driving scenarios and obtain a wider driving vision, the road conditions can be accurately predicted, greatly reducing the occurrence of various traffic accidents. The all-round three-dimensional monitoring system with driving assistance functions using high-resolution cameras is such a solution, which helps to achieve safe driving.

3D and its application in automobiles

    3D is based on people's perception of space and objects, and has depth and full perspective. 3D allows people to have a more intuitive and interactive experience. Generating 3D images requires a sophisticated Graphic Display Controller (GDC), which in turn requires a geometry unit and a structure processing unit. Integrating these elements into a graphics engine provides optimal performance, as shown in Figure 1.

Figure 1: Graphics Display Controller (GDC) Block Diagram

    Many of the best graphics controllers available today can handle both 2D and 3D graphics. However, in many cases, system designers do not fully exploit the existing 3D capabilities that can bring many benefits to end users. For example, if a driver wants to know if a flat tire or a light is not working, applying 2D technology would require hundreds of megabytes of pre-built 2D graphics.

    All these requirements and more can be easily solved with less than one megabyte of image and geometry data using 3D technology. As a leader in this technology, Fujitsu has over a decade of experience in the embedded graphics market, designing, developing and helping customers integrate leading-edge 2D and 3D graphics display controllers.

Figure 2: Using 3D images, a single object can be rotated to any angle, scaled to any size, and highlighted at any location.

    By using the graphics chip's geometry engine, this model can now be animated in any way and positioned from any angle. As long as you have the 3D model (polygon mesh and texture mapping), you don't need to provide any other data to display the 3D model in all its glory. By scaling, rotating and translating along the three axes of the 3D model, an unlimited amount of image data is available. As mentioned earlier, any amount of information and prompts can be presented, such as tire data, faulty lights and door conditions, without the need for expensive flash or system memory.

Choosing the right GDC

    In order to achieve the best display effect and image smoothness, it is particularly important to choose the correct and appropriate graphics display controller.

    From 3D rendering to image deformation, the functions of GDC are now presented to users through a variety of applications. High-end image display controllers can create images that dazzle consumers, and other levels of GDC can display information clearly and simply, allowing users to see the information they want at a glance. GDC can be divided into the following three categories based on its cost performance: low-end is QVGA screen, pre-rendered graphics, and may include image input function; mid-range is WVGA screen, mainly 2D dynamic graphics, can also support 3D, and has image input function; high-end is SXGA or higher resolution screen, dynamic 3D graphics, multiple image input.

    The first step to achieve perfect image function in the application is to select a suitable GDC for the application target and obtain the required functions at a reasonable price. It is worth noting that the automotive industry is a relatively cost-sensitive application field. For system developers, the most important task is to reduce the cost of components (BOM). For low-end to mid-range GDCs, developers can use system-on-chip (SoC) graphics controllers to meet the requirements, but due to the limited internal VRAM memory capacity and various system bottlenecks (such as bus speed), the image functions, flexibility, pixel fill rate, and screen size supported by these GDCs are relatively limited.

    When performance is the priority and cost is not so important, such applications can use high-end GDCs with multi-chip architectures, relying on external automotive microcontrollers to manage CAN transmission operations, power supplies, and peripheral components such as stepper motor controllers. In addition, since these GDCs do not have built-in VRAM and program flash memory, external VRAM can be used to support high-performance operations. In the future, the use of built-in VRAM can further reduce the cost of high-end automotive GDCs.

Developing an all-round three-dimensional monitoring system using GDC

    The car's all-round stereoscopic monitoring system uses the latest GDC MB86R11/MB86R12. MB86R11/MB86R12 is equipped with an ARM CortexTM-A9 CPU, and integrates a graphics engine corresponding to OpenGLES2.0 and various peripheral interfaces in a single chip. By performing 3D image processing on 4-way input images, high-quality driving scene maps can be drawn, and it will help developers realize more ambitious and complex 3D applications in the future. The existing software and hardware structure of Fujitsu Semiconductor's all-round stereoscopic monitoring system is shown in Figure 3.

Figure 3: Hardware and software architecture

    Let's take a look at the omnidirectional stereoscopic monitoring system implemented using the above GDC. This system is a technology that performs 3D synthesis of images from cameras installed in four directions of the car. As a visual aid for the driver, the car is equipped with a synthesis system for images from four cameras, as shown in Figure 4. However, previous technologies can only perform 2D image synthesis, and therefore can only display at a specific perspective; projecting the photographic projection image onto a 2D plane can only show a top view from above, and sometimes it is difficult to distinguish the surrounding vehicles and pedestrians. The omnidirectional stereoscopic monitoring technology can synthesize images from four cameras onto a 3D model, so that the omnidirectional scene can be displayed from any perspective; it can project images onto a three-dimensional curved surface, arbitrarily change the viewing angle, and fully show the scene you want to see, thereby improving recognizability.