Tesla applies for patent to obtain fleet data to train self-driving neural networks

According to Electrek, Tesla has applied for a technology patent, planning to obtain data from its huge fleet to train autonomous driving neural networks .

Tesla applied for the patent, but Andrej Karpathy, Tesla's head of artificial intelligence and autonomous driving software, was named as the sole inventor on the patent application.

In the patent, Kapathy describes the problem of collecting data for deep learning training in applications: "Developing deep learning systems for applications such as autonomous driving by training machine learning models.

Often, the performance of deep learning systems is limited, at least in part, by the quality of the training set used to train the model. In many cases, significant resources are invested in collecting, curating, and annotating the training data. The effort required to create the training set can be significant and often tedious. Furthermore, it is often difficult to collect case-specific data for machine learning model improvement.”

Tesla's approach to developing its self-driving system is very different from that of most other companies. While most other companies use relatively small fleets of test vehicles to collect data and test their systems, Tesla uses the tens of thousands of cars it sells to collect data. These cars are equipped with an array of sensors to collect on-road and driving data, as well as to test the operation of the self-driving system in "shadow mode."

The data collected by the fleet is very valuable for Tesla to train its autonomous driving neural network . However, it must be carefully collected and fed into the autonomous driving neural network .

"As machine learning models become more complex, such as deeper neural networks, the need for large training datasets increases accordingly," Kapathy said in the patent application.

These deeper neural networks may require more training examples than shallower neural networks to ensure they generalize well. For example, a neural network may be trained to be very accurate on the data in question, but the network may not generalize well to predict future cases. In this case, the network may benefit from additional examples included in the training data.”

Thus, Kapathy explains his patented method for sorting potential training data at the source before it is transmitted:

"An example method includes receiving a sensor and applying a neural network to sensor data. Applying a triggering classifier to an intermediate result of the neural network to determine a classifier score for the sensor data. Based on at least a portion of the classifier score, determining whether to transmit at least a portion of the sensor data over a computer network. When determined to be 'positive,' the sensor data is transmitted and used to generate training data."