The integration of radar and vision opens a new era of advanced intelligent driving
To join the "Smart Car Expert Automotive Radar Industry Exchange Group", please add 18512119620 (same as WeChat), and note company-name-position to join the group
Recently, JAC Group held a centralized display of its core intelligent technology achievements during the celebration of its 60th anniversary. During this period, Geometry, a partner working hand in hand with JAC Group, also had a "show of style" with its overall intelligent driving solution built around the "vision + 4D millimeter wave imaging radar" dual-dimensional heterogeneous information fusion perception technology route.
How this technology route is currently developing, what is the technical logic behind it, and whether it can become a new breakthrough solution for OEMs in the future have become the focus of heated discussion in the current industry.
Ray (4D imaging) visual fusion:
A new option with all-weather and all-day capabilities
In recent years, the discussion on the "pure vision" perception route has remained hot, but at the same time, facing the trend of intelligent driving gradually evolving towards a higher level, the multi-sensor fusion perception technology route has become more and more a consensus in the industry. Although each type of sensor has its own advantages, it has to be admitted that the performance of a single sensor in complex scenarios has certain limitations, especially high-level intelligent driving requires vehicles to have the ability to cope with extreme weather and all-weather complex environments. Only by using the information complementarity and iterative development of different sensors such as cameras, millimeter-wave radars, lidars, and ultrasonic radars can the robustness and safety of the perception system be improved.
How to combine these sensors to achieve the best effect? The current mainstream solutions in the industry are mainly the following two: one is "vision + lidar + ultrasound", and the other is the combination of "vision + 4D millimeter wave imaging radar".
As a high-tech enterprise and specialized enterprise deeply engaged in intelligent driving technology, Geometry Partners has been adhering to the mission of "providing safe and effective methods for large-scale mass production of intelligent/autonomous driving and connected vehicles" for 5 years since its establishment. Starting from solving the pain points such as perception bottlenecks faced in complex intelligent driving scenarios, it has developed a high-resolution 4D millimeter-wave imaging radar, combined with vision, and designed a "vision + 4D millimeter-wave imaging radar" two-dimensional heterogeneous information fusion perception technology route to create an intelligent driving solution with differentiated advantages of " multimodal, all-weather, low-cost, and easy to mass produce ", achieving better performance and better driving experience. A series of high-end intelligent driving functions such as city NOA , intercity NOA, memory driving, memory parking, and autonomous valet parking have successfully entered the vision of many OEMs.
Among them, the result of the joint development of Geometry Partners with JAC Group based on its needs - a high-level driving assistance system with urban NOA as the core function , based on the 11V6R system solution and high-level intelligent driving algorithm and all-round safety redundancy strategy, which can not only cope with conventional NOA scenarios, but also safely and efficiently handle complex working conditions such as left-right turn games, mixed traffic of people and vehicles, avoiding vehicles cutting in, special-shaped obstacles, and ghost heads. In addition, the system expands the adaptability of intelligent driving application scenarios to all-day and all-weather range, and can realize driving and parking functions in various complex weather conditions such as rain, snow, fog and haze, bringing users a better urban NOA experience.
▲
Geometry Partner's 11V
6R
sensor-based intelligent driving solution was unveiled at JAC Group's Technology Open Day
From the industry's perspective, 4D millimeter-wave imaging radar is releasing powerful vitality and opening up broad market prospects with its all-day, all-weather, low-cost and easy mass production characteristics. Faced with the automotive industry's requirements of "reducing costs and increasing efficiency" and L3/4 intelligent driving all-weather and high robustness, 4D millimeter-wave imaging radar has become a bottom-line sensor. This is not difficult to explain why 4D millimeter-wave imaging radars with cost advantages and excellent perception performance are increasingly favored by OEMs.
The "nerve network" of the technical route: system architecture
If the perception module is the eyes and the planning module is the brain, then the system architecture is the neural network that closely connects the autonomous driving system to the vehicle. From the design of the technical route to the final solution product implementation, the system architecture plays a vital role in this process.
As a supplier of integrated system solutions for intelligent driving software and hardware, Geometry Partner Intelligent Driving System is designed with high reliability as its core, ensuring the stability of the system in different environments. Whether it is day or night, sunny or rainy, the system can maintain stable operation and provide users with a comfortable, safe and reliable intelligent driving experience.
Specifically, in terms of sensor configuration of the system architecture, vision and 4D millimeter wave imaging radars are mainly used, and sensor configuration and domain controllers can be flexibly selected according to different scenarios and functional requirements . For example, in the high-performance version of the intelligent driving solution, the system design uses Orin/MDC as the main chip and 11V6R as the sensor configuration solution, realizing the refined perception and multi-modal heterogeneous information fusion of two-dimensional sensors within the 360° long and short distance range, helping to create an integrated driving and parking function.
得益于平台化和可扩展的设计优势,几何伙伴智驾系统展现了高度的泛化应用能力, 该系统设计采用面向应用的SOA软件架构 , 实 现了分层化和模块化设计,使得功能扩展和系统维护更加便捷,无需对整个系统进行大规模的改动,从而显著提高了开发效率和系统的可维护性。同时,基于无图或轻图方案,该智驾系统都能够提供精准的导航和决策支持,确保驾驶的安全和效率。
相较于“视觉+激光雷达”或是“纯视觉”的系统架构设计方案, “视觉+4D毫米波成像雷达”可作为L3/4 智驾 在极端情况下 的托底感知系统, 为系统架构设计提供了一种成本效益更高、全天候性能更好、感知精度更准确、泛化能力更强、更易于维护和升级的智驾解决方案,具有显著的市场竞争力优势。
Exploring the "code" behind the technical route
With the "vision + 4D millimeter wave imaging radar" dual-dimensional heterogeneous information fusion perception technology route and system architecture, the advanced driving assistance system with urban NOA as the core function created by Geometry Partners has won the support of many OEMs such as SAIC, FAW, JAC, Xiaomi, and Jidu. Behind this, there are four key "codes".
One of the secrets lies in the independently developed high-resolution 4D millimeter-wave imaging radar.
4D millimeter wave imaging radar is one of the core sensors independently developed by Geometry Partners to meet the complex needs of autonomous driving scenarios. With the rapid development of the autonomous driving industry, the industry has put forward higher requirements for the perception performance of automotive radars in terms of resolution, detection accuracy and reliability. As one of the sensors, traditional millimeter wave radars have been "stretched" due to their industry pain points such as low resolution, insufficient height measurement capability, sparse point clouds, and weak static target detection capabilities.
Based on this, the company, based on the all-day and all-weather perception advantages of traditional millimeter-wave radar, can output more than 60,000 high-quality point clouds per second through optimization of radar waveform design, antenna array design, radar parameters, signal processing algorithms, etc. , so that millimeter-wave radar has imaging capabilities ; combined with other sensors such as vision to perform dual-modal heterogeneous information feature-level fusion, it greatly improves the accurate recognition of dynamic and static targets in various complex scenarios all day and all weather, and the perception capability has been qualitatively improved.
Let's look at a few scenarios. The first is stationary vehicle recognition and "ghost peeking" recognition: it can accurately identify faulty/stationary vehicles in front of the lane, and more accurately identify pedestrians/non-motor vehicles "ghost peeking", leaving more decision-making and braking space for AEB in advance , improving AEB performance and improving driving safety.
The second is high- and low-dimensional obstacle recognition: based on the perceived data, it can accurately identify the entire environment around the vehicle, including high-altitude obstacles such as elevated signs and street lights, and low-altitude obstacles such as speed bumps, ground cracks, cones, and manhole covers, thereby improving the safety of intelligent driving.
The third is to realize the SLAM (simultaneous localization and mapping) function in indoor and outdoor scenes based on the 4D millimeter wave imaging radar point cloud, which can greatly improve the accuracy and robustness of mapping and positioning, and introduce semantic map information such as parking spaces to further improve the accuracy of parking space detection during parking.
The second secret lies in BEV + Occupancy.
Based on the hardware sensor configuration, Geometry Partner also matches an intelligent and efficient algorithm architecture solution to realize high-level intelligent driving functions such as urban NOA. The system architecture builds a general perception model based on the cutting-edge technologies of BEV and Occupancy, extracts point cloud information from 4D millimeter wave imaging radar and visual semantic information through feature fusion, and realizes cognitive understanding of dynamic and static environments.
相较于其他的架构,该架构创新性的采用 360°雷(4D)视BEV特征融合+规则的混合技术 ,显著提升动态障碍物的感知能力;针对不在白名单内的障碍物感知问题,进一步采用 雷视融合Occupancy框架 ,能够智能地识别和分类各种障碍物,确保系统在面对白名单外的障碍物时,依然可以做出快速而准确的响应, 显著提升全域通用障碍物的感知性能 。
The third secret lies in the end-to-end integrated network.
Based on the BEV+Occupancy technical solution, the company has realized the end-to-end integrated network technology research and development and implementation of perception, prediction, decision-making and planning, ultimately creating a more comfortable and intelligent urban NOA experience.
In terms of perception detection, the fully self-developed Geometry Partner General Perception Detection Network GGPNet can detect dynamic and static whitelist obstacles, general obstacle detection, online mapping, and traffic sign recognition. This enables the intelligent driving system to accurately identify and understand various obstacles and traffic signs in the surrounding environment, providing a solid perception foundation for autonomous driving.
在预测、决策和规划方面, 全自研几何伙伴 预测决策规划一体网络GPDPNet ,预测具有多模态、强交互和低时延的特点,使得预测更加灵活,规划轨迹更加实时和可用。相比传统基于规则的方法,GPDPNet能够覆盖更多的corner case,显著提高了自动驾驶系统的鲁棒性和适应性。
The fourth secret is to seamlessly replace ultrasonic radar.
For ultrasonic radar usage scenarios such as obstacle collision and space parking perception, Geometry Partners has conducted a large number of tests and verifications, and 4D millimeter wave imaging radar can perfectly replace them, achieving a wider detection range, smaller coverage blind spots, and more accurate detection accuracy. With the support of high-level intelligent driving algorithms and all-round safety redundancy strategies, it can seamlessly replace ultrasonic radars , realize space parking detection and close-range high-precision collision warning functions in parking mode, and provide a solid technical foundation for the safe operation and efficient decision-making planning and control of autonomous driving vehicles in complex environments.
In terms of cost, thanks to the low-cost advantage of 4D millimeter-wave imaging radar, Geometry Partner's integrated advanced assisted driving system can further reduce costs compared to other solutions, creating a more cost-effective intelligent driving solution for customers.
It is undeniable that as a disruptive innovative technology, autonomous driving still has many technical challenges to overcome, but with the support of "password" and the continuous iteration of technology, the cost-effective intelligent driving system of Geometry Partners will accelerate the full coverage from mid-to-high-end to economical models , promote equal rights for intelligent driving, realize the social value of intelligent driving, and allow more ordinary consumers to enjoy advanced intelligent driving functions.
An "accelerator" for the mass production of high-end intelligent driving
At present, products based on the "vision + 4D millimeter-wave imaging radar" dual-dimensional heterogeneous information fusion perception technology route have brought new ideas and more possibilities to the field of intelligent driving because of their advantages in line with current market demands.
Under the brand strategy of "leading intelligence with intelligence", a number of passenger car models under JAC Group are also jointly developing intelligent driving solutions based on this technology route with Geometry partners, and have formed long-term exchanges and cooperation.
As the attention to this technology route continues to increase and the technology of Geometry's partners becomes more mature, in addition to SAIC, FAW, JAC, Xiaomi, and Jidu, more and more OEMs are choosing to work with Geometry's partners to jointly explore the best implementation solutions for advanced intelligent driving functions such as Urban NOA.
We should take a long-term view. We believe that with the verification of many OEMs, the technology route of 4D-vision fusion will be an "accelerator" to promote the mass production of high-end intelligent driving. Geometry partners will continue to leverage their own technological advantages to help more OEMs realize the popularization and application of high-end intelligent driving functions and promote the transformation and upgrading of the automotive industry.
Those who travel far will eventually see the stars; those who seek light will surely welcome the dawn...
Founded at the end of October 2018, Geometry Partners is a high-tech and specialized enterprise integrating the research and development, manufacturing, sales and ecological construction of autonomous driving software and hardware products. It brings together outstanding technical experts and teams from the fields of automotive sensors and autonomous driving at home and abroad.