Apple's new car trend: using algorithms to improve processing power

Current autonomous driving systems need to process a large amount of data. The data is collected from a variety of sensors installed on the car, which can provide a full view of the road. It is very expensive from a computing perspective to understand this data, determine which objects deserve more attention, and respond to changes in the road layout.

One way to solve this problem seems to be to increase the processing power of the autonomous driving system, but the cost and energy consumption will increase due to the increase in hardware required. Another way is to let the driving system make choices and see where to invest resources, so that the number of cycles can be deliberately reduced in some places, and the saved computing power can be put on important elements.

Documents recently disclosed by the U.S. Patent and Trademark Office show that Apple seems to want to develop "depth perception sensor data processing" technology. The patent talks about how the system selects and processes sensor data.

According to the description, the sensor data processing system receives data from the car's sensors and then displays the environment in depth in the form of data. The system can have one or more passive sensor devices (such as cameras) to generate images and map the entire environment, which includes both image data and depth data, so that a basic understanding of "what is nearby" can be formed to form a preliminary model.

Then, one or more active sensor devices generate data, and more intensive hardware (such as lidar) kicks in to compare it to the model. The model is then adjusted many times with more active sensor data until it is "confident" that the model is accurate enough to feed information to the rest of the self-driving car.

How much does the algorithm correct each time it is iterated? Apple will judge the confidence level based on this correction. In short, it will continue to adjust the model with sensor data until the adjustment becomes negligible.

This approach not only saves resources and reduces costs, but also improves performance. When facing the road, the system can build models faster, so that the system can recognize objects and elements at a farther distance and recognize them earlier, thereby speeding up the response and making the system safer.

Apple also hinted that machine learning or deep learning algorithms can be used to optimize model adjustment and object recognition. In addition, multi-vehicle sensor systems can be used to expand the field of view of environmental monitoring and extend the effective "vision" of the car.

Apple applies for many patents, but concepts do not necessarily translate into reality.

Apple's self-driving project is named "Project Titan", which is basically based on computer vision and transportation. Everyone initially thought that Apple would launch its own brand of cars, but later Apple changed its focus and turned to self-driving car systems. Apple is currently testing self-driving systems in California.

Another sensor patent suggests that Apple seems to want to use lidar sensors and proximity sensors to automatically capture points of interest for drivers, such as pictures of a certain location, or scan the environment. For travelers, this feature is quite practical if they want to take images of both sides of the road; of course, it can also capture photos when an accident occurs, which may be useful for insurance companies and law enforcement agencies.

In addition, Apple is also thinking about how to use sensors under the car to monitor the speed and angle of the ground relative to the car's movement. This information can tell the autonomous driving system whether the car is slipping or not moving in the intended direction.

There are also patents introducing gesture technology that can move the car. Apple wants to use AR screens to display road obstacles on the windshield; there is also vehicle communication technology, so the car can communicate with other unmanned driving systems; and through an iPhone or similar mobile device, you can call and pay for an unmanned taxi.