Instant Localization and Mapping (SLAM) and Self-Driving Cars: The Design and Implementation Journey of DSP

When the car was first born, it was only regarded as a means of transportation to quickly transport us from one place to another; but as such a revolutionary invention, it has subverted the way humans understand the world, and it is natural that they will also enjoy it. The support of continuous innovation.

The installation of intelligent tools makes cars smarter, starting with driver assistance applications such as anti-lock brakes and power steering, and now the rise of self-driving cars with the help of artificial intelligence (AI). Tracing back to its roots, one of the key attributes that autonomous vehicles (Autonomous vehicles) must possess is the ability to track positioning and movement.

Car GPS technology, which emerged in the 1990s, is still in use today, and motion tracking is now a relatively simple task. But as a technology, it opens the door to a variety of positioning and routing applications. However, we must also acknowledge the limitations of GPS. Its accuracy is only a few meters, which makes it unsuitable for scenes where small or micro-distance movements need to be tracked, and in environments where GPS satellites are unavailable, such as cities or mountains with high-rise buildings. The difficulty of data acquisition has also sentenced GPS to death.

As cars become more automated and begin to have the ability to sense their surroundings, the need to track subtle movements becomes inevitable. We should broaden our horizons and look for solutions beyond GPS.

figure 1

SLAM has a wide range of uses. For example, in map building applications, SLAM can be used to identify your orientation in the environment. If the application determines that you are facing northwest at an intersection, it can tell you whether to turn right or left next. GPS calculations only know that you are currently at an intersection, but not your direction.

SLAM gives devices positioning awareness, and embedded vision applications have made significant progress as a result. Based on SLAM, a car can not only track the heading or direction (orientation), but also sense its movement within the environment (position, speed, and height).

The data required for SLAM operations is usually only obtained by the camera sensor, a process called visual real-time localization modeling (VSLAM). In recent years, as more sensors have become available, SLAM has evolved to fuse data from different sensors.

The working principle of the SLAM system is to track a series of feature points through continuous image frames and other sensor data, triangulate the 3D position of the camera, and use this information to simultaneously estimate the orientation of the camera (or other sensors). As long as each frame of image has enough feature points to track, the orientation of the sensor and the structure of the surrounding physical environment can be quickly interpreted.

For example, during the car design process, SLAM will call single or multiple forward cameras, radar, lidar, and inertial measurement units (IMUs, which provide accelerometer and gyroscope data for estimating sensor orientation) to determine the car's orientation. Movement mode in the current environment. When GPS data becomes available, it will also be used to verify that previous location estimates are accurate.

Figure 2 shows the various sensors installed on the vehicle body, including cameras, radar and lidar. These sensors can be used as data sources for SLAM.

Figure 2: Self-driving cars use a variety of sensors and cameras to sense their surroundings

SLAM is key to a variety of applications in driver assistance systems and autonomous vehicles. Including and not limited to:

As shown in Figure 3, the size of the SLAM-related application market is expected to exceed US$2 billion in 2024 [1]. The main driver of growth is the evolution of SLAM algorithms and growth in multiple market segments. Continuous technology development and increasing awareness of the advantages of SLAM are the main driving forces for the increase in market demand. Rising interest in the technology from industries such as autonomous vehicles and augmented virtual reality has also contributed to the global adoption and expansion of SLAM.

Figure 3: The market related to SLAM technology is expected to exceed US$2 billion in 2024

More than that, SLAM is being used for navigation applications in both indoor and outdoor environments, further driving mass adoption of this technology across multiple end-user industries. Over the past five years, leading technology companies have significantly increased their investment in SLAM and are considering it as part of their business expansion strategies such as new product development and mergers and acquisitions.

Figure 4 shows the general process of SLAM.

Figure 4: SLAM processing flow chart

The implementation of each block in the figure is based on traditional computer vision. The difference is that they rely heavily on a variety of different linear algebra and matrix operations, undertake heavy computing tasks, and can be implemented on CPU or GPU.

For general use and prototyping, the CPU is a good choice, but its performance and capabilities are limited. First, the number of single instruction multiple data (SIMD) channels used for parallel processing is small; second, the power consumption efficiency is not high. These two conditions make the CPU not the best choice for large-scale applications, and even in some scenarios, the CPU cannot meet the real-time requirements of SLAM.

GPU has taken the computing power to a new level. It has multiple modes suitable for parallel processing to help achieve high performance and meet immediacy needs. But everything has two sides. GPUs are energy-hungry beasts that consume huge amounts of power. Moreover, it is unrealistic to convince SoC suppliers to add additional components required for GPU in the limited layout planning space just by relying on this processing advantage.

The time has finally come for dedicated DSP to show off its capabilities. DSPs are highly programmable and small in size, making them ideal for mass production of devices deployed in multiple market segments.

Cadence ^® Tensilica ^® Q7 DSP 是为了实现 SLAM 在边缘及其他设备的高性能而专门设计的。Vision Q7 DSP 已经是 Tensilica 家族的第六代视觉与 AI DSPs 产品。 Cadence 对它的指令集进行了优化，使其拥有更高的矩阵运算、特征提取和卷积运算性能，是迄今性能最高的视觉 DSP，并具备在边缘设备运行 SLAM 应用所必须的高性能与低功耗间的完美平衡。

图 5 展示了 Cadence ^® Tensilica ^® Q7 DSP 的架构和关键特性。

图 5：Tensilica Vision Q7 DSP 架构

Tensilica Vision Q7 DSP 的高级特性如下：

此外， Vision Q7 DSP 的设计已通过 ISO 26262 认证 ，是汽车应用的绝佳平台。下方是将与多种传感器连接的 Vision Q7 DSP 用于 SLAM 运算的典型架构图。

图 6：使用 Vision Q7 DSP 的 SLAM 系统架构示意图

Vision Q7 DSP 也可以用于实现多种去中心化和分布式的系统。DSP 可被布置在靠近传感器的位置，在到达 CPU 之前对数据进行处理，缓解对存储器带宽的需求，同时减少数据传输量。这一方法通常被用于汽车等复杂系统，以满足新一代应用的安全关键和高性能需求。

除了已经获得 Tensilica Xtensa ^® Xplorer 开发环境的全面支持，Vision Q7 DSP 还可以充分利用已经非常成熟且进行了高度优化的 Cadence Xtensa Imaging Library 图像库。 受到 OpenCV（C++语言计算机视觉库）的启发，Cadence 将多种 OpenCV 函数进行了移植，且沿用了类似的函数名称和 API，让 OpenCV 移植过来更加平滑、容易。

Vision Q7 DSP 也已经获得 Tensilica Neural Network compiler 神经网络编译器的支持。 Tensilica 神经网络编译器将神经网络结构图编译成 Vision Q7 DSP 可执行、高度优化的高性能代码，这些代码充分利用了优化神经网络库中丰富的函数集。

基于单摄像头数据源，Cadence 在公司内部做了 VSLAM 的实现测试，并获得了 SLAM 流水线上不同区块在 Vision Q7 DSP 和 Vision Q6 DSP（Q7 的前代产品）的相关数据（见图 7）。

图 7：Vision Q7 DSP 与 Vision Q6 DSP 的速度对比：在 SLAM 不同功能块上均有 2 倍提升

对比 Vision Q6 DSP，Vision Q7 DSP 的测试结果在 SLAM 流程管线的不同功能块 均 显示了接近 2 倍的性能提升。 优化的指令集，更优的封装，以及数量更多的 MACs 让摄像头位置预估的精度更高，使用 Vision Q7 DSP 加速 SLAM 相关应用的体验也更好。 除了性能提升，Vision Q7 DSP 占用的面积和 Vision Q6 DSP 完全一致，且能耗更低，是未来产品的理想选择。