Get to know Pony.ai and decode the technology chain of this autonomous driving unicorn

Pony.ai’s Robotaxi has been further integrated into public life.

If you take a stroll on the streets of Yizhuang, Beijing, you will find that Pony.ai’s Robotaxi has been further integrated into our lives.

Following Guangzhou, the autonomous driving unicorn recently expanded its Robotaxi service to the public in Beijing, with approximately 150 stations covering the 150 square kilometers core area of ​​Beijing Economic and Technological Development Zone. 

The way to experience it is also very simple - download the PonyPilot+ APP, choose a station nearby, and wait patiently.

As long as it is within the operating hours (8:30 am - 10:30 pm), no matter it is rush hour or smog or heavy rain, your Robotaxi will pick you up safely and take you to your destination.

With the opening of the previous policy pilot zone, Robotaxi connections between Yizhuang and Daxing Airport are also expected to be realized.

With safety as its top priority, Pony.ai currently retains safety officers as the last line of defense.

In addition, Pony.ai will also connect to travel platforms such as Hongqi Intelligent Driving and T3 Travel to further expand the user range that the Robotaxi service can reach.

1. The latest PonyAlpha X system

So, how is such a smooth Robotaxi riding experience achieved?

Pony.ai co-founder and CTO Lou Tiancheng revealed that they iterated the software system more than 8,000 times last year alone; the hardware system also underwent a new upgrade.

Specifically, it is Pony.ai’s latest generation of L4 autonomous driving system - PonyAlpha X.

Its unique features can be summarized into the following two sections:

System perception has no blind spots

Through 4 lidars, 4 millimeter-wave radars, 7 cameras, and 2 GNSS antenna modules, the PonyAlpha X system can achieve 360-degree perception without blind spots. 

Moreover, this generation of systems is better at handling long-tail and complex scenarios.

For example, a new traffic light recognition camera has been added to help Robotaxi identify the status of traffic lights more accurately; a new lateral millimeter-wave radar (on the left) has been added to monitor the road conditions of vehicles going straight on the left when the vehicle turns right.

These improvements can bring passengers a more comfortable and safer experience to a certain extent.

Standardized production

Of course, whether an autonomous driving system is capable of providing services to the public cannot be judged simply by the number of sensors it has - it also has to go through a variety of "experiences".

The entire system, from design, development, production to testing and verification, refers to automotive-grade standards.

There are as many as 200 quality inspection items, involving functional testing, environmental testing and stress testing, such as vibration, high and low temperature, waterproofing, noise, actual road testing, etc.

In order to standardize and unify the software and hardware configurations of each system and move towards productization, Pony.ai has built a standardized production line for L4 autonomous driving systems.

Prior to this, Pony.ai's autonomous driving system was basically built by hand.

With this production line, the loading speed of PonyAlpha X is 6 times faster than the previous generation - this is also one of the prerequisites for the Robotaxi equipped with this system to be quickly offline and put into large-scale operation.

At present, the models equipped with the PonyAlpha X system that have been put into mass production are all Toyota's Lexus RX450h models, which are also the Robotaxi operating models of Pony.ai in Beijing Yizhuang and Guangzhou Nansha.

In the near future, Pony.ai’s Robotaxi will enter the lives of citizens in Shanghai and other cities.

2. The technical chain behind autonomous driving

As Robotaxi operations expand in Beijing, the autonomous driving technology chain behind Pony.ai has also been showcased to the public.

hardware

Almost all autonomous driving companies have gone through the era of 64-line mechanical lidar, and Pony.ai is no exception.

Due to the large size of mechanical lidar, it was usually mounted directly on the roof of the car in the early days.

The relevant person in charge of Pony.ai said jokingly,

"Because of the 'big flower pot' on top of the test vehicle, passers-by thought we were generating wind power."

In just two or three years of development, Pony.ai has undergone three changes in its adaptation of lidar, and the size of the lidar it uses has become smaller and smaller, making it easier to integrate it into an integrated system.

Including LiDAR, Pony.ai has also independently developed 24 core hardware modules, including an on-board computing platform, traffic light recognition cameras, sensor gateways, etc., to achieve standardized production in the production line.

Compared with the hardware equipment sold on the market, self-developed hardware modules will undoubtedly be better adapted to their own systems.

High-precision maps and positioning

I believe everyone is very familiar with and often encounters such a situation: the driving/walking navigation position updates slowly, or even does not update for a period of time and then suddenly "drifts".

This situation would be fatal if it occurred in an autonomous vehicle.

According to a relevant person in charge of Pony.ai, in order to ensure the safe driving of autonomous vehicles, the data provided by high-precision maps must be: precise (accurate to the centimeter level), fresh (updated in real time and on a daily basis), and wide (covering as many types of data as possible on the road).

With high-precision maps that meet the requirements, positioning must also keep up.

In addition to lidar and high-precision cameras, Pony.ai also uses GNSS antenna modules, IMU inertial measurement units, wheel speed sensors and other equipment to ensure that the autonomous driving system can still operate normally even if a single sensor fails.

Perception and prediction

Autonomous driving vehicles need to avoid obstacles while driving, which involves perception and prediction.

Sensors such as lidar and high-precision cameras can act like "eyes", identifying the position, shape, type, speed and direction of surrounding traffic participants, and predicting the movement trajectory and intention of these objects, so as to avoid some possible dangerous scenarios in advance.

For example, if the system recognizes a pedestrian ahead and predicts based on his or her direction and gait that he or she is about to cross the road, the autonomous vehicle will slow down and give way in advance.

Pony.ai said that the valid data perceived by the vehicle during driving will be transmitted back to the background, and the deep learning network will mine, automatically label and train the road test data to drive the rapid iteration of the perception algorithm and prediction module.

Planning and Control

The driving trajectory of an autonomous vehicle usually needs to be planned in advance, and Pony.ai divides it into two categories:

1) Route planning (navigation planning from point A to point B). Currently, most of the Robotaxi vehicles open to the public are fixed point-to-point connections, which fall into this category;

2) Motion planning (trajectory planning within a short period of time/distance in the future), which can be simply understood as the avoidance route when the vehicle encounters obstacles during driving, such as overtaking on a narrow road.

Control is to calculate the control instructions to complete the planned trajectory.

It is reported that in order to better plan and control, Pony.ai has also created a decision-making and numerical optimization system that will model uncertainty, integrate interactions with other objects, consider factors such as safety and traffic regulations, and try to provide a comfortable and efficient riding experience.

Infrastructure and Data

Infrastructure is the technical foundation of the autonomous driving system, and data is the fuel that makes this system run.

Pony.ai's in-vehicle system architecture mainly includes three parts: heterogeneous computing, high-performance data communication, and safety and redundant systems; the offline infrastructure includes data annotation platform, big data storage and analysis, large-scale simulation, large-scale machine learning training, etc.

Among them, big data and large-scale simulation are the key factors driving the rapid iteration of algorithms. A technical closed loop has been formed from collecting data from large-scale road tests, to big data analysis, to algorithm improvement, to large-scale simulation, and then back to large-scale road testing.

R&D testing

Of course, before an autonomous vehicle can finally become an operational product that can be put on the road, it must undergo rigorous testing.

It is reported that Pony.ai has established a relatively mature quality system internally, including defect prevention, quality management from a product perspective, and standardization activities.

In terms of defect prevention, Pony.ai actively cooperates with third-party quality suppliers (responsible for design verification testing and system testing) and has built its own hardware testing laboratory (responsible for engineering verification testing and integration testing).

With the joint efforts of both parties, autonomous driving vehicles are able to go on the road for road operation tests, and then carry out a series of tasks such as problem analysis and location, problem classification, simulation verification, and regression automation.

Pony Intelligence

As a major new business of Pony.ai, the technical details and future blueprint of Pony.ai were also disclosed for the first time.

In terms of sensor configuration, the smart card includes 2 laser radars, as well as millimeter-wave radars and high-precision cameras, which can also achieve 360-degree perception without blind spots.

The long-distance camera has a visual distance of 300 meters, which can better detect the road conditions ahead on the highway; the rear-facing camera can help the vehicle change lanes more safely in high-speed scenarios.