Kankan Intelligence-Nationally produced multi-mode ecological intelligent cockpit system based on optical imaging

1.Technology name:

A nationally produced multi-mode ecological intelligent cockpit system based on optical imaging

2. Technical description:

The Kankan smart cockpit system relies on domestic industrial chain resources and takes advantage of Kankan's optical imaging technology and product advantages to provide a centralized and complete smart cockpit solution to comprehensively meet users' multi-scenario needs and create a "future third space" that integrates leisure, entertainment, and office. The system integrates multi-modal intelligent innovation interaction, multi-scenario/multi-device intelligent ecological integration, full-scenario visual imaging technology, computational optical image algorithms, fully digital driving and other intelligent technologies to achieve non-contact and non-contact palm vein recognition authentication, voice-vision-gesture multi-modal fusion human-computer interaction, intelligent perception monitoring (IMS) of in-vehicle environment/human status, seamless migration of mobile phone-car tasks/content and multi-screen interactive display, intelligent network applications, electronic rearview mirrors (CMS), assisted driving and other functional applications, aiming to create a human-centered driving experience.

3. Unique advantages:

The main innovative technical points of the Kankan smart cockpit system include: domestic technology, multi-mode fusion interaction technology, in-vehicle intelligent perception monitoring (IMS), human-machine fusion applications, multi-screen interactive cockpit display, intelligent network connection applications, etc. Create a people-centered driving experience and promote the transformation of automobiles from a "single means of transportation" to a "third space" that integrates leisure, entertainment, office and other functions. 1. Industrial innovation In order to break through the "bottleneck" problem of key links in industrial development and dependence on foreign core technologies and products, Kankan Intelligence relies on domestic industrial chain resources and follows the trend of domestic substitution to realize a domestic smart cockpit system, including domestic computing platforms, domestic display components, domestic Camera products, domestic dual-light fusion, independent intellectual property algorithms and applications and other key technologies. Build a domestic safe and controllable smart cockpit solution to realize the implementation and large-scale development of end-to-end products of domestic components. 2. Application innovation: In terms of application innovation, the main innovative technical points of the Kankan smart cockpit system include: domestic technology, multi-mode fusion interaction technology, in-vehicle intelligent perception monitoring (IMS), human-machine fusion application, multi-screen interactive cockpit display, intelligent network application, electronic rearview mirror, etc. Create a people-centered driving experience and promote the transformation of automobiles from "single means of transportation" to a "third space" that integrates leisure, entertainment, office and other multi-functions. 3. Architecture innovation: Under the trend of automobile intelligence, electrification, networking and "software-defined cars", the originally distributed computing units will develop towards centralized development, and functions such as body, cockpit, driving, and chassis domain controllers will be migrated to the central computing platform, and the corresponding wiring harnesses will be shortened, greatly saving costs. At present, it is known that OEMs, new energy vehicle manufacturers (Tesla, etc.) and Tier1 with complete independent design capabilities from chips to complete vehicles have begun to lay out in this regard. Based on its advantages in core optical imaging technology and industrial chain integration, Kankan Intelligence designs the overall smart cockpit system architecture based on the overall trend of smart cockpits and actual product needs, and has innovative advantages in product architecture: 1) Centralized system architecture design. Combining domestic computing platforms, domestic display components, domestic camera products, domestic dual-light fusion, domestic panoramic sound systems and other technologies, we aim at platform-level overall solutions from the beginning of the design, realize system centralized design, improve integration, and effectively reduce system complexity and system cost. Using a domestically produced smart cockpit domain controller platform, integrating multi-channel camera products, microphone arrays, and the original MCU is highly integrated into the domain controller, combined with algorithms, and integrated innovation is carried out to effectively reduce the overall system cost; 2) Edge computing/end-side computing improves system efficiency and reduces computing power requirements. Based on the effective coordination of domain controller and end-side device computing, the system efficiency is improved and the computing power requirement is reduced. Based on the in-depth accumulation of Kankan Intelligence in optical design, camera modules, basic imaging algorithms, etc., relevant calculations are effectively calculated on the end side to reduce the computing power requirements of the domain controller, including 3A and other algorithms are optimized and debugged on the module side to ensure high-quality image quality and stability. The end-side AI SoC is deployed to complete the AI ​​algorithm on the end side, etc.; 2) The imaging algorithm is highly integrated to achieve software-defined hardware. Based on the combination of multiple algorithms such as imaging algorithms and fusion algorithms, the built-in functions of the domain controller, such as ISP, are fully utilized to achieve the most efficient use of resources, reduce the demand for hardware, and change the system architecture of the original vehicle-mounted camera products that mostly use external ISPs, reduce costs and improve image quality; 3) Multi-domain controller fusion innovation architecture. Realize the fusion of multi-domain architectures of cockpit domain, driving domain, and control domain, realize the fusion of multiple devices, and realize functional innovation. For example, the cockpit domain uses the Xinchi computing platform and the auxiliary driving domain uses the Horizon platform to give full play to their respective advantages and achieve fusion.

4. Application scenarios:

The project application scenarios are as follows: 1. Natural interaction - in-vehicle multi-mode fusion interaction (gesture recognition, voice interaction, AI environment intelligent recognition): Kankan technology is based on ordinary RGB camera sensors and microphone arrays. Combine gesture recognition, voice, expression and other interactive technologies to achieve an innovative and natural interactive experience. Through the combination of gesture recognition and voice and other technologies, application control (such as gesture control navigation, control music switching, answering calls, etc.), air conditioning, door control, etc. are realized. It has obvious advantages in recognition accuracy, recognition speed and cost, and truly realizes natural and efficient interaction experience; 2. Innovative interactive applications of multi-mode fusion technology combined with in-vehicle scenarios: Through the fusion of multi-mode technologies, including full-duplex voice interaction, AI visual intelligent recognition environment, facial expressions & emotions, gestures, simulate the way people communicate naturally, so that machines can understand the information that people in the cockpit want to express like humans, and quickly identify user intentions. 3. Intelligent perception and monitoring in the car (IMS): Combined with the driver monitoring system (DMS), occupant monitoring system (OMS), rear entertainment system (RSE), and streaming media rearview mirror, the cockpit intelligent perception and monitoring are realized, and applications such as safety monitoring, entertainment office, and interaction are realized. Combined with the cameras installed in the center console/A-pillar/rearview mirror, B-pillar/rear row, combined with image algorithms (distortion correction, HDR, noise reduction), and intelligent algorithms (intelligent recognition), identity authentication (personalization) is realized, and intelligent recognition of human status (driver fatigue, distraction, phone calls, etc.), items (leftovers after getting off the car, pets, children detection), intelligent photography, video calls and other safety and consumption functions are realized to complete the closed loop of intelligent monitoring in the smart cockpit. 4. Multi-screen interaction and ecological integration in the smart cockpit: 1) In-vehicle display applications that present multi-screen, AR-HUD, mobile phone projection and other display linkage and ecological integration in the car with intelligent linkage of multiple screens. Under the trend of large and multi-screen in-vehicle, the space in the car is effectively utilized to provide corresponding display solutions according to different users and different application scenarios. AR-HUD displays important information such as navigation and speed on the windshield in front of the driver in the form of optical projection; 2) Multiple screens on the car can be linked and communicated with each other while maintaining their own characteristics, such as dragging and switching between the co-pilot screen and the central control screen, which is suitable for the co-pilot to drag the central control screen to help find navigation information, order songs, etc. and then drag it back to the central control position, and for example, the front and rear passengers can share movies and TV shows synchronously. 3) Combined with the Internet of Vehicles, the smooth integration and switching of mobile phone ecological content and car-machine ecological content can be achieved. In view of the usage habits of most users and the fact that most files are completed on the mobile phone, the mobile phone content can be projected to the car screen to continue to edit files synchronously, and the unfinished movies, songs and audiobooks that users like can be played; remote interconnection and control applications can be achieved. 4) Human-machine fusion applications: Intelligent identification of the status of drivers and passengers (emotions, fatigue level, etc.) to achieve personalized intelligent adjustment. For example, when it is recognized that the user is tired and sleepy, it is intelligently suggested to adjust the seat angle downward, play relaxing music/music that meets personal habits and preferences, and intelligently adjust the air conditioner to the appropriate temperature according to the current environmental weather and user habits. 5) Identity authentication by contactless palm vein recognition: A new palm vein recognition experience, through the second generation of biometrics - palm vein recognition to achieve applications such as door opening, starting, and payment. The palm vein recognition system uses an infrared CCD camera to obtain palm vein images, compare and authenticate with the pre-stored palm vein feature values, and then authenticate the individual. This technology is suitable for unlocking car doors, extracting personal private files, custom shortcut keys, etc. It has the advantages of contact-free, hygienic, and can replace car keys in most scenarios. 6) Electronic rearview mirror (CMS) On-board electronic rearview mirror (CMS) and assisted driving system. The product of the electronicization of traditional rearview mirrors can provide a more flexible field of view and further eliminate blind spots. In addition, in overcoming bad weather scenes (such as rain, snow, night, etc.), combined with night vision and intelligent algorithms, it can provide clearer pictures and restore important driving information for drivers. 7) Computational optics: Using computational imaging phase recovery algorithms to help industrialize new technologies derived from computational imaging: By combining new diffractive optical design with AI image processing, a specially designed diffractive optical element is added to the pupil of the traditional imaging system to perform phase encoding (prior information) on the object being photographed, and then an intelligent image algorithm is used to decode and calculate the resulting intermediate image to restore the original image. This technology has obvious technical advantages in solving depth of field and image blur problems, improving full-frame clarity, real-time acquisition of ultra-massive information, and identifying key information.