AGV robot: Theoretical basis of visual obstacle avoidance

The role of the erosion operation is to eliminate meaningless boundary points in the image that are smaller than the structural element, so that the boundary of the target object shrinks inward; the role of the dilation operation is to fill the empty points of the target object in the image, so that the boundary of the object expands outward; the composite operation of erosion and dilation is called opening and closing operations: the opening operation is the process of corroding the image first and then dilating it, which can eliminate the edge burrs and isolated spots in the image; the closing operation is the opposite of the opening operation process, filling the holes and cracks in the image. They can perform simple smoothing on the image and detect singular points in the image. According to the binarization processing results, we need to remove the holes and burrs in the image and keep the original image features unchanged, so the image can be opened to make the black edges clear for edge detection.

(4) Navigation marking line edge detection algorithm

Edge refers to the part of the image where the local brightness changes significantly. It is a collection of points in the image where the grayscale of pixels is discontinuous or the grayscale changes dramatically. The purpose of edge detection is to identify points in the digital image where the brightness changes significantly. Although the processing method does not obviously rely on edge detection as a preprocessing, edge detection is still an important feature that image segmentation relies on and an important basis for image analysis. Commonly used edge detection operators are:

(4.1) Gradient operators: Sobel operator, Prewitt operator.

(4.2) Operators based on the zero-crossing point of the second-order derivative of the image function: LOG operator, ny operator.

Obstacle recognition research:

The choice of obstacle recognition method depends on the surrounding environment and the definition of obstacles. Obstacles can be defined as objects of a certain volume on the road ahead of the vehicle. Common obstacles on the road include vehicles, cargo, and debris.

The most critical technologies in obstacle recognition are detection, tracking and positioning. Detection refers to confirming whether there are obstacles on the front visual path, tracking refers to describing the trajectory of the selected target, and positioning refers to calculating the actual distance between the obstacle and the automated guided vehicle. Among them, detection is the basis, tracking is the process, and positioning is the ultimate goal.

Tracking of space targets is the process of building a template based on the effective features of the target and finding the candidate region position that is most similar to the target template in the image sequence, that is, determining the trajectory of the target in the sequence image. In the research on the problem of tracking space obstacles based on monocular vision, there are generally two ideas:

(1) Without relying on any prior knowledge, obstacles are detected directly from the image sequence and then the target of interest is tracked.

(2) Relying on prior knowledge of obstacles, the possible targets are first modeled, and then the targets matching the model are detected in real time in the image sequence and then tracked.

The second approach is the most commonly used, because obstacles exist in a specific operating environment and can be represented by a complete set of finite elements. For this tracking method, the first step to achieve tracking is target detection, that is, extracting the region of interest from the background image from the sequence image.

In the process of target tracking, it is often necessary to use a search algorithm to predict the location of a target at a future time in order to narrow the search range. Based on this idea, there are generally two types of algorithms:

(I) Predict the possible position of the target in the next frame of the image, and then find the optimal point in this relevant area. Commonly used prediction algorithms include Kalman filtering, extended Kalman filtering, particle filtering, etc.

(ii) Algorithms that reduce the target search range, such as the mean shift algorithm (MeanShift algorithm), the continuous adaptive mean shift algorithm (Camshift), and the confidence region algorithm, by optimizing the search direction and using certain estimation methods to optimize the iterative convergence process of obtaining the distance between the target template and the candidate target, thereby narrowing the search range.

The research on spatial target positioning algorithms mainly focuses on obtaining the distance of each point on the target in the scene relative to the camera, which is one of the main tasks of machine vision and the ultimate goal of obstacle recognition. By calculating the distance parameters between the target and the camera, we can get the speed of the target relative to the car, the size of the target and other parameters, and better provide decision data for the control operation status. Here I have collected several implementation ideas for visual mobile obstacle avoidance:

There are many commonly used computer vision solutions, such as binocular vision, depth cameras based on TOF, depth cameras based on structured light, etc. Depth cameras can obtain RGB images and depth images at the same time. Whether based on TOF or structured light, the effect is not ideal in strong outdoor light environments because they need to actively emit light and are easily disturbed by strong light; depth cameras based on structured light emit light that generates relatively random but fixed spot patterns. After these spots hit the object, the positions captured by the camera are different due to different distances from the camera. Then, the offset between the spots in the captured image and the calibrated standard pattern at different positions is calculated, and the distance between the object and the camera can be calculated using parameters such as camera position and sensor size. For AGV, binocular vision is more suitable:

The distance measurement of binocular vision is essentially a triangulation distance measurement method. Since the two cameras are in different positions, just like the two eyes of a person, they see different objects. The same point P seen by the two cameras will have different pixel positions when imaging. In this case, the distance of this point can be measured through triangulation distance measurement. The points calculated by the binocular algorithm are generally image features captured by the algorithm, such as SIFT or SU features, and the sparse graph is calculated through the features.

The key to obstacle detection based on binocular stereo vision lies in two points: ① Extraction of obstacle targets, that is, identifying the position and size of obstacles in the image; ② Stereo matching points between pairs of obstacle target area images, so as to obtain the depth information of obstacle targets. The former step is the basis of the latter step. There can be multiple targets identified. Only after the disparity is obtained by stereo matching can we mark which targets are obstacle targets.

The realization of binocular stereo technology can be divided into: image acquisition, camera calibration, feature extraction, image matching and 3D reconstruction. The optical axes in the above figure are approximately parallel. In the parallel optical axis system, binocular visual ranging transforms the problem of finding the depth of the target in the 3D scene into the problem of finding the parallax in the 2D projection image. Therefore, the camera model is to establish a one-to-one mapping relationship between the points of the 3D scene and the points on the 2D image.

Review editor: Liu Qing