TI: mmWave sensors enable intelligence at the edge

Aguilera

TI: mmWave sensors enable intelligence at the edge [Copy link]

Intelligent processing at the edge with mmWave sensors can reduce the amount of data sent to central servers and increase the amount of decisions made by the sensors themselves. The Internet of Things (IoT) is driving more devices and sensors to connect to the network in building and home systems: Gartner estimates that the number of devices covered by the IoT reached 8 billion in 2017. But as the number of sensors connected to the cloud increases, system requirements for network bandwidth, remote storage and data processing are also rapidly increasing. Intelligent processing at the edge can reduce the amount of data sent to central servers and increase the amount of decisions made by the sensors themselves. This can improve system reliability while reducing decision latency and network costs; if the server is down, the last thing you want is for your sensors to not be able to detect objects and make decisions! Edge Intelligence and Connectivity Millimeter wave (mmWave) sensors enable edge intelligence in two ways. First, mmWave provides unique data information such as distance, speed and angle, while also having the ability to reflect different targets, which allows sensors to detect specific characteristics of different objects in the detection range. For example, velocity data allows sensors to see micro-Doppler effect- Modulation effects from tiny movements - which contain typical features of target objects, such as the spinning spokes of a bicycle wheel, the swinging arms of a walking person, or the running limbs of an animal. The system can use this data to classify and identify the type of object in the sensor's field of view. Reducing false detections Second, mmWave sensors enable edge intelligence through on-chip processing. Sensors containing microcontrollers and digital signal processors (DSPs) are able to perform primary radar processing, as well as feature detection and classification. Figure 1 shows the results of an experiment using on-chip intelligence in a 50-meter outdoor intrusion detector for a security application. Intrusion detectors are used to determine if a person has entered a protected area, such as a shipping yard, parking lot, or backyard. Some sensors that rely on optical or infrared sensing may detect false motion from nearby trees and shrubs. Instead, mmWave sensors use processing and algorithms to filter out and prevent false detections, triggering the detector only when there is human motion. Security cameras and video doorbells can perform the same false detection filtering by connecting to a network server to process images. These server-based systems typically provide features that require a user fee, but mmWave technology enables decision making to be done in the sensor itself without the need for a networked server. Figure 1: Example of on-chip filtering for a long-range outdoor intrusion detector Figure 2 shows intrusion detection using mmWave technology; the mmWave sensor analyzes the velocity of objects in the scene, filters out motion from the moving background, and tracks only the person.

Figure 2: Animated point cloud from an outdoor intrusion application. Black dots represent moving objects, including people, trees, and bushes. The algorithm displays people as green while filtering out other moving objectsFigure 3 shows the difference in the micro-Doppler signatures of a walking person and an oscillating fan. Once the correct features to separate the two objects are identified, the classifier makes the distinction in real time on the device.Figure 3: Two images showing micro-Doppler information over time for a walking person and an oscillating fanFigure 4 shows how on-chip processing enables the mmWave sensor to identify and classify objects based on their signatures in real time. These features can be based on size, reflectivity, micro-Doppler effect or other characteristics, and can help identify typical behaviors to distinguish different moving objects. For example, classification can be used to identify people and animals in indoor or outdoor security applications, distinguish between children and adults in home automation systems, or determine whether a person is running or walking in a restricted area. Figure 4: Example of using mmWave sensors to perform classification: All moving objects are assigned a track in the middle image, and the colored areas represent people. Edge processing and intelligence can be powerful tools to help improve the quality and robustness of IoT sensors and networks. mmWave sensors with integrated processing can enable intelligence at the edge to address the problem of false detection by filtering and classifying objects, more intelligently identifying what is happening in the scene and making decisions in real time. - Keegan Garcia is a marketing manager at TI, responsible for advancing new applications of mmWave sensors in smart infrastructure and factories. He has hardware application experience with TI multi-core DSP processors and has supported high-speed interfaces such as DDR3, SerDes and PLL.