360 hacked another Tesla, using N ways to interfere with sensors

       In last week's preview of Defcon, Cheyunjun mentioned that a team from 360 will demonstrate how to deceive, or interfere with, the sensors to make the semi-autonomous driving system on the car make the wrong decision.

       This Monday, Liu Jianhao, head of the 360 ​​Automotive Information Security Laboratory, together with Xu Wenyuan, a professor/doctoral supervisor at Zhejiang University, and Yan Chen of Zhejiang University, who were also conducting research on this topic, gave a demonstration at Defcon, introducing how to fool the ultrasonic sensors, cameras and millimeter-wave radars on the Tesla Model S.

       Autonomous driving, including the current semi-autonomous driving, is achieved by sensing the environment around the vehicle through various sensors installed on the vehicle. The data is transmitted to the analysis and processing unit. The control unit makes a judgment based on the results sent by the analysis and processing unit, and then sends commands to the vehicle's actuators to make different commands such as steering, acceleration, and braking.

       Their entry point is that in the first step, when the sensor perceives the surrounding environment, they do some tricks to make the sensor obtain wrong data.

How to jam the sensor?

       Different sensors interfere in different ways. The three sensors studied in this study work on different principles. Let's look at them one by one.

1. Ultrasonic sensor

       The function of ultrasonic sensor on the vehicle is to detect obstacles. After sending out ultrasonic waves, the distance to the obstacle is determined based on the time it takes to receive the echo. Tesla's Model S has 12 ultrasonic sensors installed around the body to detect obstacles around the vehicle. It will issue a warning sound after detecting an obstacle, and the corresponding position of the obstacle and distance prompt will be displayed on the dashboard. If Autopilot is turned on, the vehicle will automatically brake when the distance is too close.

       The ways to influence ultrasonic sensors are very simple. One is to emit ultrasonic noise to increase the signal-to-noise ratio of the echo received by the ultrasonic sensor to affect its judgment; the other is to emit ultrasonic waves of the same frequency to the ultrasonic sensor, making it mistakenly believe that there are obstacles where there were originally no obstacles.

       It is also very easy to implement. It only requires a jammer costing 60 RMB to send a corresponding signal in front of the ultrasonic sensor of the vehicle. According to the results of the team's research, if the interference is through noise, it will affect the ultrasonic judgment of the distance, and the judgment result will be inaccurate, so the distance information displayed on the dashboard will also be wrong; and if the signal of the same frequency is sent, the sensor will make a completely opposite judgment.

After the interference, the sensor misjudges the distance

       If the driver is driving manually, he will be misled by the error message displayed on the dashboard. If the driver is in Autopilot mode, he will also be misled and make wrong judgments, or mistakenly think that the system is not working and issue a warning sound, and the human driver must take over. It should be noted that if the same frequency signal is sent to interfere, then the time is very important. Only the first ultrasonic wave received by the sensor is effective and can have an impact.

The sensor failed to detect an obstacle

       Of course, in addition to this, there is also a simple way, which is to use some sound-absorbing materials to absorb the ultrasonic waves emitted by the sensor and directly strike it.

2. Millimeter wave radar

       In comparison, it is much more difficult to crack the millimeter-wave radar. The most direct manifestation is the cost of the equipment. According to Liu Jianhao, the cost of their equipment to interfere with the millimeter-wave radar is 1.2 million yuan . However, due to safety considerations and the limitation of the length of the power cord of the equipment, the interference with the millimeter-wave radar is not achieved when the vehicle is running at high speed.

       The theoretical interference distance of millimeter-wave radar is the detection distance of millimeter waves. However, there are few wireless millimeter-wave transmitters and the cost will be higher, which is why attacks on millimeter-wave radars will be more difficult to occur in reality.

Millimeter wave radar jammer

       The above picture shows the device that interferes with the millimeter wave radar. Liu Jianhao said that through electromagnetic interference, the sensor can be made to think there is a car when there is no car ahead. This information will be displayed on the dashboard synchronously. If the distance is close enough, the alarm device will also remind you. Similarly, in the Autopilot state, it will automatically brake. Similarly, when there is a car ahead, the system can be made to think there is no car ahead and continue driving.

Incorrect display of the instrument panel after interference with the millimeter wave radar

       Similar to ultrasonic radar, interference equipment can also be used to cause the system to make incorrect distance judgments. Regardless of whether it is ultrasonic radar or millimeter-wave radar distance judgment, in actual driving scenarios, only a small distance error will be more likely to mislead people, such as when reversing or parking, and more interference is for the Autopilot system.

       If the sensor is severely interfered with, the system will also make the judgment that "the sensor cannot work properly" under Autopilot, and Autopilot will not be able to be started, requiring manual mode to be entered.

3. Camera

       Although the cause of the fatal Tesla accident in May has not yet been determined, the "blinding" effect of the truck's white container on the camera is exactly what Liu Jianhao started to study. However, the interference with the camera was not done directly on the car, but the camera was removed.

Blinding Camera

       The method of blinding a camera is to use a light source to shine directly on the camera (or directly on a calibration plate and let it reflect light towards the camera). The final effect depends on the distance between the light source and the camera, as well as the intensity of the light source. According to Liu Jianhao, using 200 milliwatts of infrared light at a distance of 50 cm from the camera can blind the camera for 40 seconds. If the distance increases, the effect will decrease. If the light source is too strong, it will cause the camera to be directly burned.

The result of the camera being blinded

       Of course, in real life, a distance of 50cm is not possible for a moving vehicle. As the distance increases, the intensity of the light source can be increased to cause blindness. And as long as the blindness lasts for 2-3 seconds, it is likely to bring serious consequences to a car in automatic driving mode.

       From a hardware perspective, the camera itself has a refresh rate. If the refresh rate is high enough, the blinding effect will be reduced.

What does it mean that sensors can be tricked?

       From the previous description, we can see that attacking from the sensor is not an easy way to implement.

       Among the three sensors mentioned, ultrasonic is the easiest to implement. The equipment is low-cost and easy to get started with. A power bank can keep it working. Although there is a distance limit, it can also be achieved by following the car in addition to the basement. As for blinding the camera, on the one hand, the farther the distance, the worse the effect. Although blinding for even one second may cause serious consequences, it requires careful planning, especially for millimeter-wave radar.

       The reason why we started with sensors is that 360 discovered the impact of sensors receiving erroneous data on the system during the process of studying autonomous driving. However, interfering with sensors is not the fundamental purpose. Liu Jianhao said, "We are still studying autonomous driving. Interfering with sensors is just to show that there are still defects in the models and algorithms of autonomous driving at this stage, which need to be solved." Model S was chosen because Autopilot can be turned on when the vehicle is stopped, which is convenient for experimental research. Tesla has also contacted 360, and the two sides will jointly study this issue.

       After the sensor was interfered with, the system obtained the wrong result. From Liu Jianhao's point of view, on the one hand, the sensor itself has no "elasticity" mechanism, and on the other hand, there is no fault tolerance in the decision-making algorithm. Taking the millimeter-wave radar as an example, the frequency and wavelength of the Model S millimeter-wave radar are reversed. If the elastic mechanism is set and the frequency is random, then the difficulty of reverse engineering will naturally increase; and from the algorithm level, although the Model S currently performs data fusion of different sensors, there is no analysis process for abnormal data.

       Before Liu Jianhao went to the United States, Che Yunjun watched their real-time demonstration of ultrasonic interference in 360's underground garage. In addition to the wrong judgment made by the wrong information, another problem was discovered:

       Under normal conditions, after turning on Autopilo, if the vehicle is moving forward, the presence of an obstacle behind it will not affect the vehicle's state. However, if the interference device is placed near the rear sensor (about 3 meters is enough), the vehicle will automatically stop. When the vehicle is reversing, interfering with the front ultrasonic sensor can achieve the same effect.

When reversing, the front ultrasonic sensor is interfered with, and Autopilot automatically brakes

       The reason for this result is not yet known, but it is known that it lies in the judgment mechanism of the Autopilot system itself.

360 and autonomous driving

       A question that comes to mind is, why did 360 start researching autonomous driving? In this regard, the idea of ​​360 Offense and Defense Lab is similar to that of George Hotz, the hacker who founded Comma.

       According to Liu Jianhao, they are currently converting a hybrid car into an autonomous vehicle. However, the means of realization are different from those of other teams. Currently, those who achieve autonomous driving related functions through modified cars all add actuators to make the actuators electronically controlled. However, the technology of actuators is monopolized by first-tier suppliers.

       Liu Jianhao believes that if autonomous driving is to be popularized, it can be achieved through software modification. As long as the steering, power and brake controls of the vehicle can be electronically controlled, control can be obtained through the reverse actuator control protocol. Then, sensors can be added to turn the vehicle into a vehicle with L2 assisted driving function.

       Liu Jianhao did not reveal much about the current modification progress. Theoretically, this method is feasible. Although there are many problems in the actual operation, such as the process of reverse control protocol, and they need to study the execution control of the vehicle for a long time, this method also brings new ideas to the automotive industry.

Car Cloud Summary

       At the end of the discussion, it seems that "360 is going to study autonomous driving" is more interesting to Cheyunjun than "interference with sensors". After all, the difficulty of interfering with sensors is there. But what cannot be ignored is that the safety of autonomous vehicles is what people are worried about. In addition to sensors, data transmission, V2X and CAN bus architecture are all nodes that need attention.