Endpoint AI revolutionizes embedded vision applications

Endpoint AI is the new frontier of AI, which sinks the capabilities of AI to edge devices. It brings a revolutionary way of data management: collect relevant data locally and make decisions locally. It upgrades IoT devices that were previously only used to calculate data into smart devices with integrated AI, thereby increasing the ability of devices to make real-time decisions. In short, it makes intelligent decisions based on machine learning physically closer to the data source itself. Therefore, when embedded vision sinks to endpoint devices, it is no longer just about breaking down images or videos into pixels, but about understanding pixels, understanding their meaning, and making smart decisions when specific events occur.

What is Embedded Vision?

Embedded computer vision gives machines vision, enabling them to better understand their environment with the support of machine learning and deep learning algorithms. Many industries have applications that rely on computer vision, and it has become an indispensable member of black technology. To be precise, computer vision is part of the field of artificial intelligence (AI), which enables machines to extract meaningful information from digital multimedia information sources and take actions or make decision recommendations based on it. Computer vision is similar to human vision, but there are still some differences between the two. Behind human vision is the ability to understand all kinds of different things seen. Computer vision can only recognize what it has been trained to do, and there is a certain error rate. On the other hand, embedded vision can enable devices to identify specific objects in the shortest time through training, so that massive images can be analyzed more efficiently. In this respect, machine vision is superior to human vision.

Embedded vision is widely used in smart terminals in the consumer and industrial fields to increase the added value of equipment. Here are a few simple examples: analyzing product quality on a production line, counting the number of people in a crowd, identifying objects, analyzing the content of a specific area, etc.

When implementing embedded vision applications on endpoint devices, the computing power of the device will be a challenge. However, if it is processed centrally, the amount of data transmitted from the sensor device to the cloud for analysis may be very large and exceed the network bandwidth. For example, a 1920 x 1080 camera running at 30 FPS (frames per second) may generate about 190 MB/S of data. In addition to privacy issues, the round-trip of data from the edge to the cloud and from the cloud to the endpoint is bound to introduce latency. These limitations are not conducive to real-time applications.

IoT security is also an issue that the market needs to consider in order to adopt and develop embedded vision. A key concern with using smart vision devices is the possibility of sensitive images and videos being used inappropriately. Unauthorized access to cameras not only violates privacy, but can also lead to more serious consequences.

AI Vision on Endpoint Devices

Endpoint AI can understand captured images

Endpoint AI uses machine learning and deep learning to match and identify patterns based on training

For optimal performance, AI algorithms run on the end device without transmitting data to the cloud. Data is captured by the image recognition device and then processed and analyzed in the same device.

Power consumption constraints on endpoint devices still exist, and microcontrollers or microprocessors need to be more efficient to handle the large number of multiply-accumulate (MAC) operations required by AI algorithms.

Deployment of AI Vision

There are countless use cases for AI vision applications in the real world. Below are some examples where Renesas can provide comprehensive MCU and MPU solutions, including the necessary software and tools, to enable rapid development.

Smart access control:

Voice and face recognition bring more value to security access control systems. However, real-time recognition requires embedded systems to have very high computing power and on-chip hardware acceleration. To meet this challenge, the MCU or MPU provided by Renesas has high computing power and integrates many functions that are critical to supporting face and voice recognition, such as built-in H.265 hardware decoding, 2D/3D graphics acceleration, and ECC on internal and external memory to eliminate soft errors and achieve high-speed video processing.

industrial control:

Embedded vision can be applied to multiple scenarios including safety operations, automation, product sorting, etc. Artificial intelligence can help perform multiple operations during the production process, such as packaging and distribution, ensuring quality and safety at all stages of the production process.

Transportation:

Computer vision can also improve transportation services. For example, in autonomous driving, computer vision is used to detect and classify objects on the road. It can also be used to create 3D maps and estimate movement trajectories. Autonomous vehicles use cameras and sensors to collect environmental information, and then use visual techniques such as pattern recognition, feature extraction, and object tracking to interpret the data and make the most appropriate response.

Embedded vision can be used for a variety of purposes, but all of them need to be customized and optimized for a specific field, and specially trained using the data set in that field. For example, monitoring a physical area, identifying intrusions, detecting crowd density, counting the number of people or specified objects or animals, finding people, finding cars by license plate number, motion detection, and human behavior analysis.

Case Study: Crop Pest Detection

Visual AI and deep learning can be used to detect a variety of anomalies - plant pest and disease detection is one example. The results show that computer vision can provide better, more accurate, faster and more economical solutions compared to previous methods that are expensive, laborious and slow to produce results.

The method and steps used in this case can be applied to any other test. There are three main steps:

The first step is performed on a computer in the lab, and the second step is deployed on an endpoint device, such as a node device in a farm. The result in step 3 is displayed on the user's screen. The following figure shows the general process.

in conclusion:

We are experiencing a revolution in high-performance intelligent vision applications across multiple market segments. The increasing computing power of microcontrollers and microprocessors in endpoint devices has brought tremendous opportunities for new visual applications. Renesas visual artificial intelligence solutions help you enhance overall system capabilities by providing embedded AI technology with intelligent data processing at the endpoint. Our image processing solutions for edge devices have low power consumption characteristics and support multi-model and multi-feature inference. Start developing your visual AI applications now with Renesas Electronics products and tools.