Laser Point Cloud Series 2: Problems and Countermeasures Encountered in LiDAR Point Cloud Processing

In the previous article "Detailed explanation of the processing process of lidar point cloud data ", the author mainly explained in detail the processing process of lidar point cloud on autonomous vehicles .

In the process of communicating with experts from lidar companies and downstream OEMs or solution providers, the author found that perception algorithm personnel will encounter many problems during the current processing of laser point clouds. For example, problems with point cloud noise, too many or too few point clouds, FOV design problems, problems with point cloud overlapping areas, problems with calibration parameter offsets, etc.

This article will elaborate on the problems encountered in point cloud processing and the corresponding countermeasures from two dimensions: the technical level and the engineering level.

01. Technical issues and countermeasures

1.1 Problems and countermeasures of point cloud noise

Point cloud noise refers to some invalid points collected by lidar, which can easily cause misdetection of the target detection algorithm model. There are two main sources of laser point cloud noise: one is the noise caused by the surface of the target object. For example, the properties of the surface material of the target object (high reflectivity surface materials cause the point cloud reflection energy to be too strong, making the target object larger than its actual size), roughness (the uneven surface causes the emission angle of the point cloud to change ), etc.; on the other hand, there is noise caused by the external scanning environment, such as rain, snow, fog, dust and other particles that block the point cloud and cannot be reflected back to the point cloud.

The previous article "Detailed Explanation of the Processing Process of Lidar Point Cloud Data" mentioned that the processing of point cloud noise mainly focuses on the filtering process in the pre-processing stage. Filtering processes noise points from the perspective of algorithm application, but some noise points cannot simply be processed by filtering algorithms. For example, environments such as heavy rain or snow cannot be directly processed by filtering algorithms. For another example, the filtering algorithm cannot directly handle the impact of certain electrical signals on lidar.

Next, the author sorted out several more typical noise factors, and detailed their respective problems and countermeasures in turn.

(1) Noise caused by object surface

When the laser point cloud scans the surface of some special target objects, some noise is caused due to the high energy (high reflectivity) of the point cloud reflected back from the target object.

For target surfaces with high reflectivity, laser point clouds usually exhibit high-reflection “ghosting” and high-reflection “expansion” phenomena. Among them, high-reflection "ghost" refers to the fact that lidar is very sensitive to high-intensity echoes reflected from high-reflectivity targets, which prompts the target to have a real point cloud image in the original real coordinate system. A point cloud image of similar size and shape will also appear at other nearby locations. For example, traffic signs, license plates, tail lights, etc. High reflection "expansion" means that after the laser scans the surface of a target with high reflectivity, the point cloud image will spread to the surroundings, making the original target point cloud image appear larger.

 Figure: "Ghost" phenomenon in point cloud images (data source: Sagitar Juchuang)

 Figure: The "high reflection" phenomenon of street signs in point cloud images (data source: Sagitar Juchuang)

Then, both high-reflection "ghosting" and high-reflection "expansion" will cause false detections, and both may force the vehicle to take unnecessary obstacle avoidance measures.

Point cloud noise caused by object surfaces can mainly be solved at the hardware level and algorithm level.

At the hardware level, technicians strengthened the factory testing of lidar products to improve lidar's ability to distinguish high-reflectivity objects and low-reflectivity objects.

At the algorithm level, it is mainly processed by filtering algorithms. Tang Qiang, perception algorithm engineer at Zongmu Technology, said: "The noise caused by the object surface can be removed by setting threshold conditions to remove abnormal points."

Regarding how to set the threshold conditions, a perception algorithm engineer from an autonomous driving company said: "For example, the algorithm model will first locate a dense point cloud area and calculate the average distance from each point in the area to its center point. Then set this average distance as the initial threshold condition. If the target point cloud is outside this initial threshold range, the point cloud is a noise point.”

(2) Noise caused by bad weather

Bad weather is an environmental factor that is difficult for autonomous driving systems to cope with, especially rain, snow, fog and dust. These weather conditions will cause laser point clouds to generate a lot of noise. The following will analyze the impact of these four environments on lidar point clouds in turn.

The first is the rainy environment. Raindrops are mainly crystal-like, and the laser will lose a certain amount of energy when hitting them, because the water droplets will cause some specular refraction of part of the laser beam. In addition, as the amount of rain increases, the rainfall may form a cloud of fog due to the temperature difference on the ground, which may cause the autonomous driving system to mistakenly think that there is an "obstacle" ahead.

The second is the snow environment. Snow is solid and tends to form into larger solids. In addition to obstacles that pile up into clumps, heavy snow days can also easily cause large areas of snow to form on the ground, which will be detrimental to ground point cloud segmentation processing in the target detection process.

Then there is the foggy environment. Generally speaking, when fog is not serious, such as light fog (visibility is 1km-10km), fog will not affect the processing effect of lidar point cloud. However, when the visibility in fog becomes lower and lower, the laser The transmittance of the point cloud will decrease, and the point cloud image in front of the vehicle will form an illusion similar to a blob, which will cause false detections.

Finally, there is the dust environment. Compared with the previous three, dust may be more difficult to deal with. On the one hand, dust will form clumps of objects, which can easily lead to misidentification by lidar; on the other hand, dust is different from rain, snow and fog. After it adheres to the surface of lidar, it will not dry and disappear naturally. It needs to be cleaned immediately with a cleaning device. Clean up.

So, how to solve these problems?

Yin Wei, senior manager of SAIC, said: "If lidar is only used to identify obstacles, the impact of point cloud noise will not be particularly large; if these point cloud data are used to outline free space (driving area, which refers to automatic If the driving vehicle can be planned and controlled), you need to consider using traditional filtering algorithms for processing, but the degree of processing of these noises by the filtering algorithm may not be well controlled. "

Although traditional filtering algorithms can be used for point cloud denoising, the calculation amount of this method is very large, and the final effect of the algorithm also depends on the technical level of the technician. Therefore, in the autonomous driving industry, technicians will also use neural network models to deal with point cloud noise.

A perception algorithm engineer at an OEM said: "Perception algorithm personnel can directly use deep learning models to identify obstacles in front of the vehicle and directly ignore noise such as rain."

In general, instead of using traditional methods to identify these noise points, technicians should ignore these noise points and directly use neural network models to identify obstacles ahead, such as vehicles, pedestrians, etc. After all, these obstacles are the focus of detection. Target objects, and rain, snow, fog and dust (under not serious circumstances) will not affect normal driving safety.

(3) Noise caused by electromagnetic signals

Nowadays, with the increasingly developed 5G network, various mobile phones, laptops and other electronic products are everywhere. As a precision instrument, lidar will be interfered by the electromagnetic waves generated by these electronic products , resulting in noise. In addition, when self-driving vehicles pass through certain areas, such as airports and power generation, these areas will also produce electromagnetic wave interference.

Regarding the reasons why electromagnetic signals cause noise, Leon, head of systems and applications at Tudatong, said: “There are mainly two reasons: first, electromagnetic signals will affect the entire circuit of the lidar, such as capacitors, etc.; second, electromagnetic signals It will affect the receiving end of the lidar, because the sensitivity of the receiver is very high, which means it is more susceptible to interference from electromagnetic signals.”

In order to explain the reason why electromagnetic signals affect the internal circuit and receiver of lidar, an expert from a lidar company said: "After a strong electromagnetic signal is transmitted into the lidar, it causes a voltage change (more than The rated voltage inside the lidar part) affects the normal operation of the lidar."

Based on the above reasons, the noise caused by electromagnetic signals is essentially due to interference on the hardware side of the lidar. Therefore, this kind of noise cannot be handled by filtering algorithms. To correctly solve the noise caused by electromagnetic signals, the key is to rely on the quality of the lidar's own hardware.

Leon said: "In the early development stage, lidar manufacturers will do some EMC testing ( electromagnetic compatibility testing), which requires shielding circuit testing in different frequency bands."