A multi-level map construction algorithm suitable for dynamic scenes

Figure 3. Point cloud maps and octree maps. The top row shows the dense point cloud maps constructed using the ORB-SLAM2 algorithm and the dense mapping module. The second row shows the dense point cloud maps constructed using the previous method [9] as the localization module, which excludes the point clouds located in the potential moving object detection area. The third row shows the dense point cloud maps constructed by our algorithm. The bottom row shows the octree maps generated by our algorithm.

The planar map constructed in Figure 4 accurately perceives static background planar structures in dynamic scenes. This can be applied to advanced scenarios such as augmented reality and serve as landmarks to enhance the accuracy of camera pose estimation.

Figure 4: Generated maps of repeated objects placed next to each other. The image on the left provides an overview of the sequence.

Object map construction

We evaluate the performance of object map construction on 8 dynamic sequences of the TUM dataset, as shown in Figure 5. To verify the accuracy of object map construction, we overlay the constructed object models on the dense map and project them onto the image plane. In high-dynamic scenes, our algorithm is able to accurately model almost all objects in the scene, unaffected by the different camera motion patterns and dynamic objects in the environment. However, in low-dynamic scenes, two people are constantly sitting next to the table, resulting in severe occlusion of static objects and background. Therefore, it is inevitable that our algorithm lacks sufficient observations for some objects, resulting in inaccurate modeling of some objects. The experimental results show that our algorithm is very effective in object parameterization, object data association, and object optimization strategies. By overcoming the influence of dynamic objects, the constructed object map provides strong support for subsequent applications such as semantic navigation, object grasping, and augmented reality.

Figure 5. Object map. Regularly shaped objects, such as monitors, books, and keyboards, are represented using cubes, while irregularly shaped objects, such as chairs, bottles, and teddy bears, are represented using quadratic surfaces.

Robustness testing in real-world environments

We also tested our method in real-world scenes using a Realsense D435i camera to verify its effectiveness and robustness. In the experiment, a person made irregular movements within the camera's field of view. To verify the robustness of the algorithm, we evaluated two camera motions: 1) moving from one end of the scene to the other; 2) almost stationary. The results of multi-level map construction are shown in Figure 6. The experimental results show that our algorithm is able to construct accurate dense point cloud maps, octree maps, planar maps, and lightweight object maps under different motion states of objects and cameras.

Figure 6. Multi-level map construction results in real-world scenes, where the camera moves from one end of the scene to the other in the upper set of images and remains almost stationary in the lower set of images. Images (a), (b), and (c) represent dense point cloud maps, octree maps, and flat maps, respectively. Image (d) shows a lightweight object map, where objects are superimposed on the dense point cloud map (image (e)) and projected onto the image (image (f)) to demonstrate the effectiveness of object map construction.

Dynamic object tracking experiment

We further apply the constructed object map to dynamic object tracking, using a co Neo3 device to capture scene images and use our algorithm to build an object map. In this case, the depth information of the map points is obtained through stereo matching, and these calculations are only performed on key frames to ensure real-time performance. The constructed object map is shown in Figure 7(a). Once the object map is built, the user can select the target object to track. When the user moves the object, the system uses KCF single object tracking and optical flow tracking algorithms to calculate the real-time pose of the object. Figure 7(b)-(d) show the dynamic tracking results of a book, keyboard, and bottle. The experimental results show that our algorithm can accurately model objects in a dynamic environment, provide accurate object models and poses for object tracking, and thus is very valuable for practical applications. In addition, this also highlights that our algorithm does not depend on a specific device, demonstrating its robustness and versatility.

Fig. 7. Object modeling and dynamic tracking in real scenes.

Summarize

In this paper, we propose a multi-layer map building algorithm tailored for dynamic scenes. We successfully build dense point cloud maps, octree maps, plane maps, and object maps containing static backgrounds and objects in the presence of dynamic interference, which enriches the environmental perception capabilities of mobile robots and expands the application scenarios of building maps in dynamic environments. Extensive experiments demonstrate the accuracy and robustness of our algorithm, while dynamic object tracking experiments further confirm its practicality. In the future, we plan to consider the real motion of other movable objects besides humans, and use planes and objects as landmarks to optimize the camera pose and further improve the localization accuracy.

Editor: Huang Fei