A vision-based autonomous navigation mobile grasping robot construction plan

Mobile grasping is a complex robot system composed of a mobile chassis, an arm, and multiple components. Since it is a complex system, we might as well break it down into multiple systems and then talk about them one by one.

As shown in the figure above, the mobile chassis is the "legs and feet" of the mobile grasping robot system, responsible for "walking"; the mechanical arm is the "hand" of the robot system, and this "hand" is a subsystem that integrates the arm, the end effector (usually an electric gripper), and the visual camera. The entire robot system "calls each other", integrates and collaborates, and finally realizes mobile grasping.

In the entire system of the mobile grasping robot, the mobile chassis belongs to the drive layer. The chassis itself has independent movement and can control the forward and backward movement and steering of the chassis. In addition to the basic forward and backward movement and steering functions, after connecting to related sensors, 2D-SLAM, 3D-SLAM and visual SLAM navigation planning functions can be realized. The following takes 3D-SLAM as an example to briefly introduce how the mobile chassis can realize path planning, obstacle avoidance and other functions.

▍Build

The main components list is as follows (schematic):

Refer to the figure below for the hardware installation effect. (PS: A robotic arm will need to be installed later, and the actual installation requires a reasonable layout of the industrial computer and RTK)

▍Navigation process

The navigation process first requires environmental modeling to generate an environmental map, and then uses the environmental map for positioning and path planning.

The modeling process uses point cloud P matching, integrating 3D Lidar, IMU, and GPS data to build a 3D point cloud map of the environment in real time.

After completing the modeling, specify the target point according to the built map, and the mobile chassis can perform intelligent path planning!

▍Robot arm "system"

As mentioned above, the arm that realizes grasping is a small system of the mobile grasping robot. The following will introduce how this subsystem realizes the process from target recognition to grasping.

Visual recognition can be achieved through the following method. That is, a large number of data sets containing recognition targets are pre-labeled in possible scenarios, and interference items are added to obtain a complete training set. After the sample instances of the training set are scaled and segmented according to a certain strategy, the convolution iteration training is used to obtain the feature map. These feature maps are then used to match feature anchors in the actual picture, and finally the feature edges are corrected by the bounding box regression algorithm to obtain the feature candidate area.

After the camera recognizes the target object, it sends its coordinate points to the robotic arm, ultimately achieving accurate grasping of the target object.

After the above detailed analysis of the mobile grasping robot system, we can know that the mobile grasping robot system is not a simple "material stacking", but a highly complex robot system with intelligent path planning of the mobile chassis, visual recognition, mobile chassis and grasping system, and multi-sensor fusion as the core.

In the mobile grasping robot system, the mobile chassis gives unlimited application possibilities to the traditional robotic arms that can only work at fixed points. Therefore, whether in traditional fields or in the scientific and educational market, there is no shortage of professional manufacturers working in this direction. In the industrial field, mobile grasping robots can make the application scenarios of robotic arms more diverse, which can be said to free them from fixed points, and not be limited to traditional applications such as palletizing, loading and unloading.

This is especially true in the field of science and education. As a complex robot system, mobile grasping covers multiple aspects of robot-related fields, such as robot motion control, environmental perception, navigation planning, learning, robotic arm motion planning, etc. A set of robot systems can be used by researchers to study different aspects, truly realizing multiple uses of one machine.

However, a set of mobile grasping robots requires high costs, from hardware selection to integrated installation, to SLAM algorithm optimization, and finally to truly realize mobile grasping.

"It is very expensive to develop a complete set of mobile grasping robots from scratch, and it also deviates from the research and learning purposes of many scientific research teams and student teams. Based on the ROS robot platform independently developed by our company, we have developed multiple sets of mobile grasping robots, covering a variety of mainstream robotic arms, lidars, depth cameras, etc. at home and abroad. Users can choose as needed. Individuals, research institutes, and university teams engaged in ROS research, motion control research, agricultural picking research, inspection research, and vision research can adopt ours, which can save a lot of time and cost, so that they can invest in research and development more conveniently, efficiently, and attentively." said Dr. Song Zhangjun, general manager of the Robot++ Shenzhen team.

It is reported that the Robot++ Shenzhen team (Shenzhen Shihe Robot) is a professional mobile robot manufacturer. It has been deeply involved in the field of science and education and has a keen sense of smell for the science and education market. As early as its inception, it began to invest in research and development to develop mobile grasping robots. Now it has established in-depth cooperative relationships with robot arm and sensor manufacturers such as KUKA, Kinova, Ruike Zhilian, and Suteng, and has launched a number of mobile grasping robots with multiple functions such as multi-machine collaboration, autonomous path planning, and visual recognition.

By Ken Shirriff

Reviewing Editor: Huang Fei