More than just computing power! Cabin-driver integration + AI empowerment, the era of smart cockpit 4.0 is here

With the rapid development of AI smart cockpits, multiple tracks including large models and in-vehicle voice will also benefit greatly.

At present, manufacturers with large-scale model technology such as Baidu, SenseTime, and Huawei have launched smart cockpit products one after another. In terms of in-vehicle voice, suppliers such as iFlytek, Unisound, and AISpeech have all launched related products.

Bao Qingfeng, general manager of Unisound's smart cockpit solution center, pointed out that the improvement of the end-side software and hardware platform will help accelerate the implementation of large models on the edge. At the same time, AI models are beginning to be diverted from the cloud to the terminal. The combination of end and cloud will become a new trend for miniaturization and precision processing of large models for specific mission scenarios, which will effectively reduce the requirements for end-side computing power.

It is reported that Yunzhisheng's self-developed mountain and sea large model has been implemented in multiple projects in the automotive industry.

Unisound's smart cockpit end-side big model includes a voice big model and an application big model. The voice big model processes the entire chain of voice recognition, semantic understanding, and voice synthesis on the end-side to achieve a more natural, more understanding, and more interesting interaction effect. For example, the voice recognition big model can not only support multiple languages ​​and dialects, but also greatly improve the accuracy and efficiency of recognition; the semantic understanding big model can well help the voice interaction system to "understand your natural language."

Based on the application big model, we can expand the multi-scenario and multi-modal interactive car use scenarios, including car use scenarios, travel scenarios, active care scenarios, health scenarios, etc., and provide more professional and personalized generated content for the scenarios. We can also create AI changeable personalities based on the technical capabilities of the mountain and sea big model, realize "free chat", and create users' "intimate friends".

"Our ultimate goal is to use these large models to turn the future smart cockpit into a user-exclusive smart assistant, providing users with more services and companionship. At the same time, through the provision of AI capabilities, we can help car manufacturers better operate based on users and achieve longer-term value and benefits." Bao Qingfeng said.

iFlytek launched the Spark Intelligent Cockpit OS based on its self-developed Spark big model, which includes three major parts: Spark Car Assistant, Spark Scenario Application and Spark Car APP. Among them, Spark Car Assistant is based on the deep learning and powerful semantic understanding capabilities of the Spark Car big model. By deeply integrating voice and visual modalities, it brings a more natural, free and intelligent human-computer interaction experience.

In addition, the Mars Car Assistant also has a rich knowledge base and online query capabilities, and can monitor and understand the vehicle condition in real time. It can accurately answer various questions that users may have about their cars, truly becoming an all-knowing car assistant that understands users better.

In addition, the Spark Big Model also fully empowers major high-frequency application scenarios of user travel, and creates different cabin services based on scenario fusion data, helping vehicles to achieve a more humane and immersive user experience.

The relevant person in charge of Desay SV pointed out that the rapid upgrade of the intelligent driving system will affect the interaction between users and the cockpit. More and more functions and richer scenes of intelligent driving need to be delivered to users through the intelligent cockpit. At the same time, during driving, users' demand for intelligent cockpits is also increasing.

He pointed out that the two major domains of smart cockpit and smart driving connect the major elements of people, vehicles and environment. Under the cabin-driving fusion architecture, the AI ​​cockpit will also be able to empower the smart driving experience. For example, by combining voice, visual and other warning reminders, it can achieve a deep experience of smart driving.

Freya related personnel also stated that cabin-driver integration will be able to derive many new functions and new interaction modes by combining the cockpit and ADAS applications, which will provide more added value and a safer driver experience.

For example, activating parking or influencing ADAS functions through voice or gestures can enhance the interactive experience and the overall feeling of functional connection on the one hand, and enhance the overall user experience and trust in the reliability of the functions through a more intuitive HMI display on the other hand.

Or upgrade the interactive dimension of CMS, make some annotations for the following vehicles and pedestrians, and display them through CMS; or through linkage with DMS or OMS, provide differentiated image quality under different weather and environmental conditions, create a scenario-based experience, etc.

It is reported that Faurecia is currently also developing related application scenarios.

Some industry insiders also pointed out that there is currently very little interaction between the two major areas of ADAS systems and cockpit systems, especially the interaction scenarios on the user side are very limited.

Gong Siying, senior director of technology products at Ecarx, introduced that Ecarx has introduced AI big model applications based on the cabin-driver integrated big model architecture built on the central computing platform. It not only creates a super-human AI travel assistant, but also realizes a truly human driving experience, comprehensively improving the user travel experience of smart cars.

First, the reasoning and decision-making of the intelligent driving system will no longer be based solely on the data of the ADAS perception system, but will be processed through multi-dimensional and multi-modal data integration, including perception data inside and outside the cabin, sound data inside and outside the cabin, chassis perception data, and car networking data, etc., which will be integrated for unified calculation. The regulation and control link is no longer limited to a single intelligent driving domain, but is based on a centralized central computing platform, similar to the decision-making process of the human brain, and then the chassis execution link.

For example, when there is a baby in the car, the vehicle not only considers the external perception information when making intelligent driving decisions, but also integrates and calculates the perception information of the child in the cabin, so as to adopt a more stable driving style during the autonomous driving process. It can also integrate the external perception information with the external sound information into the decision-making. For example, when encountering special scenes such as police cars, ambulances, and fire trucks during driving, the system can better optimize the execution strategies of lane changing, acceleration, overtaking, etc. during the autonomous driving process.

From the above, we can see that the next generation of smart cockpits is rapidly evolving towards high-level intelligent trends such as multimodality, scenario-based, personalized, and proactive. With the exploration of innovative applications and scenario expansion, new opportunities and challenges have arrived.