What is an automatic parking system? Analysis of automatic parking path planning and tracking technology

Figure 8 Parking trajectory and steering wheel angle of group 2 (4.5, 8.0, 0)

Figure 9 Parking trajectory and steering wheel angle of group 3 (7.6, 8.5, 0)

Figure 10 Parking trajectory and steering wheel angle of group 4 (7, 10.2, 0.3)

3.3 Robustness Analysis

In actual driving, there are certain errors in the state perception of sensors (such as inertial navigation, steering wheel sensors, distance sensors, etc.) and the speed control and steering wheel angle control of the vehicle by the controller. In order to verify that the intelligent agent has strong robustness and can resist the interference of perception and control errors, Gaussian noise is used to represent random disturbances and applied to the state s (x, y, θ, sw, d) and action a, as shown in formula (15). X represents the variable value. From the Gaussian distribution, it can be seen that the probability that the disturbance value is in the interval [-2σ, 2σ] is 95.45%.

As shown in Table 3, referring to the accuracy of the real vehicle perception system and control system, 4 different groups of random errors were set for parking experiments, and the parking trajectory and steering wheel angle curves were obtained, as shown in Figures 11 and 12. It can be seen that the different variances of random Gaussian noise and the uncertainty of generating disturbance values ​​lead to different changes in the parking trajectory and steering wheel angle of vehicles in different tests with the same variance and different variance tests. The larger the variance, the larger the maximum value of the trajectory deviation, and the vehicle needs to take a larger steering wheel angle to counter the deviation, especially when entering the parking space to fine-tune the position. This feature is particularly obvious. Although it is subject to large random interference, resulting in different states of the vehicle at the same time step, and the state after taking action is not as expected, the intelligent agent can still adjust the steering wheel angle in real time according to the current state and finally move to the target position.

Table 3 Random noise variance settings

Figure 11 Parking trajectories with different noises

Figure 12 Steering wheel angles with different noises

To further analyze whether random noise affects the parking ability of the model, 10,000 groups of initial postures were randomly generated in the A3 area, and the parking performance of different groups of errors was statistically analyzed, as shown in Table 4 (where

Calculated according to formula (16). When the random disturbance is small, the number of successful parking and the x-direction error are basically unaffected, but the heading angle error gradually increases. When the rear axle center exceeds the target position in the y direction, parking is completed, so the y-direction errors are all less than the minimum step length of 0.1 m and the difference is not large. The success rate of group 4 is 67.3%, which is a significant decrease. The x-direction error and the heading angle error increase, indicating that the interference of random noise has a significant impact on the intelligent agent. Combined with the noise variance setting, it can be seen that the increase in the heading angle noise variance is very small, indicating that the parking performance of the intelligent agent is most affected by the heading angle noise. This is because the maximum single-step change of the heading angle is only 1.2° (calculated by formula (6) when the steering wheel angle reaches the maximum value of 540°), while the heading angle noise variance of group 4 is 3°, which is equivalent to the actual heading angle change of the vehicle at each time step is much smaller than the random disturbance.

Table 4 Planning performance indicators with and without noise

3.4 Planning Capability Analysis

When analyzing the planning ability of a planning algorithm, we can consider aspects such as time consumption, success rate, path length, tolerance, parking error, and path smoothness. The tolerance refers to the minimum distance between the vehicle and surrounding obstacles during parking. The larger the tolerance, the higher the safety. Path smoothness can be measured by the change in curvature. The larger the change in curvature, even if it approaches infinity, the less smooth the path is. For a path represented by discrete points, use formula (17) to calculate the curvature of the curve between two consecutive points.

As a classic geometry-based path planning method, the Reeds-Shepp planning algorithm [RS] has the advantages of simple implementation and fast solution speed. Many experts and scholars have studied and improved RS and applied it to actual parking systems. The RS curve is the shortest path connecting the starting posture and the ending posture when the vehicle can move forward and backward, satisfying geometric constraints and fixed curvature. It is divided into 5 categories, with a total of 48 possible solutions, and each path consists of a maximum of 5 segments. Figure 13 shows the CCC (curvecurve-curve, 3 arcs) case in the RS curve, with the origin S as the starting point and G (x, y, φ) as the end point. From the geometric relationship, the equation group (20) can be obtained, where t, u, and w are the center angles of each arc segment, so as to obtain the length of each arc segment.

Figure 13 CCC type diagram of RS curve

The application logic of the RS algorithm is to solve 48 situations (possibly without solution) in turn given the starting and ending positions, perform collision checks on feasible solutions, and then use the cost function to select a path from the collision-free paths for output. Since the RS curve consists only of arcs and straight lines with the same radius, there are only 48 possible solutions for a set of starting and ending positions, and the situation is even worse if obstacles are considered, which leads to the RS algorithm's weak planning ability. In contrast, the path planning algorithm based on deep reinforcement learning consists of many small fragments, each of which has a changing curvature, and the number of solutions can be considered infinite, so it has stronger planning and generalization capabilities. Therefore, the two are compared in terms of planning ability to illustrate the superiority of the path planning algorithm based on deep reinforcement learning.

In the interval x∈［2，7.5］，y∈［7，10］，θ∈［－0.2，0.2］，a set of starting poses is randomly generated. The agent trained by the improved algorithm and the RS algorithm are tested. The results of each indicator are shown in Table 5. It can be seen that the average time consumption of the RS algorithm is 4.3 ms, and the final pose error is 0, but the success rate is only 3.0%, while the success rate of the algorithm in this paper is as high as 54.2%, and the time consumption and error are not much different. Since the curvature of the RS curve is discontinuous at the tangent point, the smoothness is much higher than that of the planner obtained in this paper based on the vehicle kinematic model.

Table 5 Planning capability indicators

The starting positions of successful parking using the two algorithms are plotted as a scatter plot (Figure 14).

Figure 14 Scatter plot of initial position of parking area

It can be seen that in the A3 area, the parking start posture of the improved algorithm covers the entire area evenly and densely, while outside A3, the distribution of the starting position is relatively sparse, because in these areas it is difficult to complete parking by reversing once, and it is necessary to rub the garage at least once, and the vehicle designed in this article has not yet considered the forward and backward switching function; the ability to park successfully is due to the offset of the initial heading angle. Even so, the parking area of ​​the improved algorithm is still much larger than that of the RS algorithm, and the parking start posture is more dense. This also shows that the generalization ability of the RS algorithm is insufficient. A certain posture can be parked, but it cannot be parked in the neighborhood of that posture.

4 Conclusion

1) The principles and characteristics of the deep reinforcement learning algorithm model were studied; a vehicle kinematic model was established based on Ackerman steering geometry; the vehicle posture, steering wheel angle and minimum distance to obstacles were used as states, and the target steering wheel angle was used as the action; an automatic parking algorithm framework based on deep reinforcement learning was built, and the operating logic of the interaction process between the intelligent agent and the environment, the sample storage process in the experience pool and the intelligent agent training process in the framework was analyzed.

2) Taking the angle between the line connecting the center of the vehicle's rear axle and the target position and the heading angle as the measure, a guided reward function is designed to avoid the convergence difficulty caused by sparse rewards; in order to cut off the temporal correlation between samples, an experience pool structure based on a priority queue is designed to store and eliminate samples with the average reward of each round as the priority, and the corresponding experience pool update process is explained; for the parking problem, a multi-stage training method based on curriculum learning is proposed, which divides the training of the intelligent agent into 4 stages, gradually expands the random area of ​​the initial posture, and accelerates the convergence of the model.

3) Combined with examples, the convergence, generalization, robustness and planning ability of the improved algorithm are studied. The simulation results show that compared with the original algorithm, the convergence speed of the improved algorithm is 25% faster, and the success rate of the two after convergence is basically the same. The planner obtained after training has strong robustness. As the variance of Gaussian noise increases, the parking trajectory offset of the agent increases, and the parking success rate gradually decreases, but the final posture is almost unaffected. Even when the extreme value of the noise in the x and y directions reaches 10cm and the distance error reaches 20cm, the parking success rate is still 67.3%. Compared with the traditional RS curve method based on geometric planning, the agent trained based on the improved algorithm has stronger planning and generalization capabilities, a larger parking area and a denser initial parking posture.