The era of sensor fusion is coming

Over time, sensors have evolved from simple analog and mechanical structures to digital devices on chips that can be connected to a machine to monitor its health and environmental conditions. Similarly, sensor fusion (multiple sensors working together to solve a problem) can be combined with many other technologies to create something new and exciting.

The idea of ​​using computing devices to sort, combine and draw conclusions from data from multiple sensors dates back to the 1950s, but it was difficult to implement. Around 1960, mathematicians developed an algorithm that tried to use machines to draw conclusions based on input from multiple sensors and filter out invalid data introduced by noise or other sources. Soon after, the military decided to adopt this technology. By processing input from multiple sources and comparing it with stored data, the military was able to better track and identify potential air targets and even calculate their exact location. With the emergence of more advanced computers and sensors, the technology has continued to improve, but there are still some problems that are too complex and require a high price.

Potential applications

Microprocessors were once mentioned as a solution when they first came out, and sensor fusion can be thought of in the same way. If you can somehow monitor multiple sensors, analyze the data in real time, and issue simple instructions or control an action, then sensor fusion can do so much more than you think! Here are just a few simple examples:

Health monitoring—including fitness, patient monitoring, and research

Elderly health monitoring - health monitoring to reduce staff burden

Automotive, transportation systems—monitoring and controlling transportation efficiency and safety

Public safety—Identify potentially dangerous situations more accurately than simple fire and security systems

Entertainment—Gaming, including controllers and virtual reality headsets

Weather - Smart weather forecast stations that not only predict changing weather conditions, but also control systems to prepare for storms (e.g. close storm shutters, shut off valves, etc.)

HVAC/air quality monitoring—intelligent control of indoor temperature, humidity, air quality, system maintenance, etc.

While all of the above capabilities have existed for years, the system’s ability to detect multiple sensors and draw intelligent conclusions, and even take action, is revolutionary.

As in other electronic fields, some IC manufacturers have also embarked on the arduous task of technology fusion. At present, various digital sensors can be effectively connected through off-the-shelf sensor fusion and sensor hub chips or other means, without having to develop algorithms by themselves.

The approaches taken by these IC manufacturers may vary, some will modify existing product lines, and some will develop new products to solve sensor fusion problems, but ultimately they will all be implemented through dedicated control chips, such as MCUs, sensor hubs, or sensor fusion processors. Now, this technology has been successfully applied to smartphones, sports monitors, and other consumer products.

The new generation of smartphones from Apple, Samsung, and others do not even require external connections, and can combine multiple powerful sensors, including a three-axis magnetometer, a three-axis accelerometer, and a three-axis gyroscope. This combination is often referred to as 9-DoF, or nine degrees of freedom.

In most cases, these features in mobile phones are "always on". If all these sensor data are processed by the central microcontroller of the mobile phone, the battery life will be greatly shortened. On the contrary, using efficient dedicated microcontroller chips as sensor hubs to process data can greatly reduce power consumption. NXP's ARM M3 series MCU is an example. EETimes reported that the results of a survey by chip analysis company Chipworks showed that Apple's M7 processor uses NXP's customized M3 chip to monitor various sensors in the iPhone5S, including gyroscopes, accelerometers and compasses. Samsung phones use Atmel's 8-bit AVR MCU.

With powerful board-level sensing technology, applications can take advantage of the 9-DoF capabilities in the phone to monitor health and exercise, or combine GPS and external data to provide users with more information. Now, external data can be added to those multi-function sensors through Bluetooth communication, with almost no restrictions. The goal of chip manufacturers is to enable engineers to design systems that can provide real-time sensor data, such systems can not only provide the required contextual information, but also use low power consumption and long battery life. In addition to smartphones, highly optimized solutions can also meet the needs of various applications such as tablets, ultrabooks, IoT devices, game consoles, healthcare, environmental monitoring and wearable computers.

Development boards that can help engineers easily experience this sensor technology are now available on the market, such as Atmel's ATAVRSBIN2. Atmel integrates fusion sensors into many products and calls it a "complete sensor ecosystem." Atmel realizes that analyzing and fusing multiple sensor data at the same time is not an independent task. In order to solve this complex problem, the company has cooperated with several well-known sensor manufacturers and sensor fusion experts, hoping to provide a complete and easy-to-implement sensor hub.

The current trend is to package three or more MEMS sensors with an MCU, such as STMicroelectronics' LIS331EB, which integrates a high-precision three-axis digital accelerometer with a microcontroller in a 3×3×1mm package. This microcontroller uses an ultra-low-power ARM Cortex-M0 and has 64K bytes of flash, 128K bytes of RAM, embedded timers, 2 I²C (master/slave) and one SPI (master/slave). The LIS331EB can also process external sensor data, including a total of nine external sensors such as gyroscopes, magnetometers and pressure sensors. STMicroelectronics' iNEMO Engine Sensor Fusion Software Suite uses a set of adaptive prediction and filtering algorithms to process (or fuse) complex information from multiple sensors. The LIS331EB is used as a sensor hub, fusing all inputs together through the iNEMO Engine software.

Freescale has also introduced a product line that packages microcontrollers and sensors together. Their FXLC95000 Xtrinsic motion sensing platform integrates a MEMS accelerometer and a 32-bit ColdFire microcontroller. Similar to the STMicroelectronics device mentioned above, the FXLC95000 can manage both internal and external sensor data. Freescale is the first company to offer an integrated sensor hub microcontroller that can also be set up according to specific applications and algorithms. This series of devices can manage up to 16 sensor inputs and can separate calibration, compensation and sensing functions from the application processor, but it needs to be used with Freescale's own or third-party drivers. 

The Freescale Xtrinsic FXLC9500 32-bit MCU sensor fusion hub with accelerometer provides a scalable, autonomous, high-precision multi-sensor hub solution that provides local computing and sensor management capabilities in an open architecture

Manufacturers such as Bosch, Fairchild, Honeywell, Microchip and TI have also made great achievements in this field.