Analysis of Transformer-based Autonomous Driving Sensor Fusion Technology

Quantitative analysis

Here, we compare the previously discussed method with nuScenes, a large multimodal dataset consisting of data from 6 cameras, 1 lidar, and 5 radars in Table 1. The dataset has a total of 1000 scenes, with training/validation/test sets split into 700/150/150 scenes.

Cameras: Each scene has 20 frames of video, 12 FPS. 3D bounding box annotation is 0.5 seconds, and each example includes 6 cameras.

LiDAR: 32-beam LiDAR with 20FPS also annotated every 0.5 seconds.

Metrics: Follow the official metrics of nuScene. The key metrics are as follows: nuScenes Detection Score (NDS), Mean Average Precision (mAP), Mean Translation Error (mATE), Mean Scale Error (mASE), Mean Orientation Error (mAOE), Mean Velocity Error (mAVE), and Mean Attribute Error (mAAE).

Some research conclusions

Accurate 3D object detection is one of the key challenges that needs to be addressed for the perception reliability of autonomous vehicles, and sensor fusion helps to leverage the strengths of all sensors on a platform to make these predictions more accurate. Transformers have become one of the most important approaches for modeling these cross-modal interactions, especially when sensors operate in different coordinate spaces, which makes perfect alignment impossible.