Understand the intelligent evolution of ST sensors and move towards the era of online life!

Can sustainable development (Sustainability) and online life (Onlife), two seemingly contradictory factors, coexist harmoniously? The first ST Sensor Conference held in Beijing on October 17, 2023 gave the answer.
 
Cao Zhiping, executive vice president of STMicroelectronics and president of China, emphasized two key words. The first is sustainable development (Sustainability), creating a sustainable world through sustainable development. The second keyword is online life (Onlife). We have entered an era where we are always connected. It is no longer meaningful to simply distinguish between online and offline. In the era of online life, the sensor and cloud connection market has really begun to explode. era. Integrating sustainable development and online life requires three core characteristics of sensors: intelligence, safety and precision.
 
Intelligent evolution of sensors
 

ST Asia Pacific Analog Devices, MEMS and Sensor Products Division (AMS) Davide BRUNO, Vice President of Marketing and Applications of the MEMS and Image Sensor Sub-Product Division and Head of the Smartphone Innovation Center

 
What is a smart sensor? Davide BRUNO, Vice President of Marketing and Application of MEMS and Image Sensor Sub-Product Department of ST Asia Pacific Analog Devices, MEMS and Sensor Products Division (AMS) and Head of Smartphone Innovation Center, introduced that the so-called smart sensor is no longer a simple embedded code processing function. , but to be able to automatically analyze data, process data at the edge, and take intervention actions based on the needs of specific applications and ensure the required accuracy, thereby making end applications richer. Current smart watches or bracelets can implement edge computing of data through sensors, improving system performance and reducing energy consumption and costs. However, this is not enough. Smart sensors also need to sense the surrounding environment. This is also the current direction of ST sensor products, allowing sensors to ultimately make independent decisions based on the data they collect and sense, and automatically take intervention actions to achieve the desired results.

The Intelligent Sensor Processing Unit (ISPU) is ST’s biggest step into the field of artificial intelligence, said Francesco BIANCHI, ST MEMS sensor product marketing manager. The ISPU is a highly specialized DSP with an ultra-low-power, high-performance programmable core that supports processing of internal (accelerometer, gyroscope, and temperature sensors) and external (connected to sensors via sensor hub) data. By running the C language algorithm compiled by the ST ISPU tool chain, you can also use NanoEdge AI Studio to generate an anomaly detection library, which is suitable for implementing any AI and sensor fusion algorithms without using an MCU. It is therefore ideal for anomaly detection, automation, asset tracking, alarms and other applications ranging from wearable accessories to high-end personal electronics.

 
Francesco BIANCHI, ST MEMS Sensor Product Marketing Manager

“Artificial intelligence cannot work without data,” Francesco BIANCHI also said. vAFE (Ad-hoc analog front end with motion detection for specific applications (verticals)) opens the door to the interconnection between artificial intelligence and external sensors. Collecting more information and original data in this way can help customers perform more tasks through data, thereby achieving system upgrades and optimization.
 
The necessity of data partition control

Data calculation and processing can be completed not only on the sensor side, on the edge side, but also on the cloud. But how to divide the labor, Francesco BIANCHI said that it is mainly determined by the complexity of the algorithm. The ST sensor can achieve an effective balance between calculated data.

In fact, as early as 2019, ST launched a sensor equipped with machine learning functions, and was the first company in the market to propose such a concept. From a system optimization perspective, if some raw data can be processed locally, more space can be left for the microcontroller to process complex tasks, thereby further optimizing the system and overall power consumption. At the same time, data partitioning methods can also help further protect data privacy. Generally speaking, most of the data collected through sensors is raw data. If traditional sensors are used, these data will be transmitted to the cloud or placed on the application side for processing. If this process is not well protected, data leakage will occur. and obtained by third parties. But if the sensor itself can process the data, it can help reduce the possibility of data leakage.

Francesco BIANCHI also said that the processing method of data partitioning is a major trend in achieving sustainable development at the industry system level. In this way, the sensor no longer passively accepts or perceives the surrounding environment, but makes intelligent judgments after sensing the environment and collecting relevant data. This is also the solution ST hopes to bring to the industry and society.

Marc VASSEUR, director of ST image sensors, added that image sensors also need to capture photons as close as possible, that is, capture pixels. There is so much data, and the development of many algorithms is very time-consuming - many algorithms that can be used by end users take more than 2 years. The reason why the algorithms provided by ST can be used by customers is to ensure that in the process of data processing and partitioning, especially in the process of image processing, it is very close to the collection of data at this level, ensuring that the sensor can achieve effective data collection and processing. and storage.

 
Marc VASSEUR, Director of ST Image Sensors

 
Opportunities for sensors in the automotive market

There is no doubt that automotive applications provide huge development opportunities for sensors, and the development of all new technologies is constantly changing the traditional car usage model. Davide BRUNO believes that automotive applications need to pay attention to two major points: first, safety. Every country has very strict laws and regulations on safety. ST can not only meet the requirements of each country, but also ensure the safety of the equipment itself. Thereby improving the safety of the car and the driving process. Currently, OEMs in the automotive industry in many countries require devices to reach the "0ppm" level. This is not easy, and ST has been committed to working with OEMs to achieve this. In addition, cars have become an extension of our lives to some extent, and many people can participate in remote video calls while driving. Especially when the car reaches autonomous driving level four or five, you can even enjoy happy times in the car, which also involves the driver and passenger monitoring system (DMS and OMS), that is, the cockpit monitoring system.

Jm LEANG, senior manager of image sensor market and technology application in ST China, added that much of the current growth potential in the automotive industry comes from DMS and OMS. DMS is mostly required by national laws and regulations. Many car manufacturers will use DMS as the main selling point to explain the safety of their products and the differences between other competing products. OMS For many consumers, the car is no longer a traditional car - the car is a mobile office and a home. There will be various large screens, small screens, and multi-screen interactions in the car, which is a human experience. an improvement. With the OMS system, the car can help calculate how many passengers there are, thereby appropriately adjusting the interior temperature, air conditioning air flow direction, etc. Therefore, OMS is a development trend in the automotive industry.
 
 

Lian Zonghai (Jm LEANG), senior manager of image sensor market and technology application in ST China

 
Currently, there are more and more new applications of sensors in cars. Vincent Gravity seats, chassis, heat pump air conditioners, keyless entry systems, vibration monitoring, occupancy recognition, posture recognition and the now popular road noise active noise reduction technology (RNC) applications all require more and more sensors. In addition, when the car is on a slope, it is difficult to open and close the door. More and more customers are also asking, "Can a sensor be added to the door control so that girls with weak strength can easily push the door?"

Vincent XU, Director of MEMS Sensor Marketing and Technology Applications, ST China

 
At the same time, Francesco BIANCHI said that for any application, if you want to further improve performance, you can consider using super sensors after multi-sensor fusion. In the automotive industry, automotive OEMs always want high-precision sensors to meet ADAS requirements. ST can use sensor fusion technology to fuse data from multiple high-precision sensors to form a super sensor to further empower ADAS performance. It is reported that this technology will be applied to ADAS next year.

 
Opportunities for sensors in the industrial market

In industrial applications, ST has witnessed how sensors can further support and enhance higher work efficiency brought about by human-machine collaboration. First of all, ST motion sensors have a very broad product line in the industrial field. In order to meet the special needs of the industrial market, ST has proposed a ten-year long life cycle project. At the same time, ST accelerometers can help better capture abnormal engine vibration data. These data are generally very small and precise. If such signals can be captured, it can further help customers perform predictive maintenance because of such abnormal vibrations. It often happens when the machine is about to break down. If the customer predicts it in advance and makes relevant corrections or adjustments, it can further reduce the damage caused to the production line and the equipment itself after the engine is damaged in the later stage. In addition, industrial inclinometers can help measure extremely small angle changes. Francesco BIANCHI said that some customers in the construction field specifically use industrial inclinometers to measure building surface settlement, deep displacement, etc.