The complexity of the Titan project may be due to Apple's uncompromising attitude towards product quality?

According to MEMS Consulting, the US Patent and Trademark Office recently published a patent application from Apple for Project Titan, an autonomous driving car project, proposing a multi-mode sensing system for nighttime autonomous driving object detection and recognition. When driving at night, the superior performance of Apple's new system compared to traditional car headlights will greatly improve driving safety and avoid traffic accidents that cause casualties.

Most of the currently designed autonomous driving systems detect and identify objects in the vehicle's surrounding environment by collecting and processing a large amount of data captured by multiple sensors. Autonomous vehicles need to process sensor data as much as possible in real time in order to achieve automatic navigation and robust control of the automated system. This is exactly what Apple's invention patent is trying to improve.

Overall, Apple's invention patent proposes a multi-modal sensing system and its application, which enables self-driving cars to detect and identify surrounding objects while driving at night.

Apple introduced in the patent that nighttime or low-light environments pose challenges to the control systems of autonomous vehicles. For example, relevant regulations limit the illumination level of vehicle headlights at night, which may correspondingly limit the effective detection range of visible light sensors (such as cameras), which are usually used for obstacle detection and identification.

Limiting the effective range of detecting and classifying obstacles (e.g., about 60 meters) will reduce the safety of autonomous vehicles and reduce the maximum speed at which the vehicle can safely travel.

Near Infrared (NIR) Sensing

A combination of complementary image sensing technologies can effectively address the challenges of object detection and classification for autonomous vehicles at night or in low-light environments. For example, current regulations may have loose or no restrictions on the illumination levels of near-infrared illuminators on vehicles.

NIR illuminators, when paired with NIR sensors, can capture high-resolution images of objects in or near the vehicle’s path and can significantly increase the effective detection range (e.g., up to 200 meters).

A larger effective detection range can enable the vehicle to detect and classify obstacles earlier while driving, improve driving safety, and indirectly increase the maximum speed of autonomous driving. NIR illuminators can project NIR light in a relatively narrow field of view (e.g., a 30-degree fan).

Although visible light sensors may have a relatively limited detection range, they can provide high-resolution image data in multiple color channels (e.g., red, green, and blue) and can also provide a wider field of view (e.g., 120 degrees).

Long Wave Infrared (LWIR) Sensing

Long-wave infrared sensors can capture the thermal radiation emitted by objects in the vehicle's surroundings, so they do not need to rely on illuminators. The effective detection range of long-wave infrared sensors is mainly limited by the sensor resolution and the resolution requirements for object detection and classification.

Long-wave infrared sensors and arrays can provide a wide 180-degree field of view around the vehicle. Long-wave infrared sensors can provide relatively low-resolution images of objects in the vehicle's surroundings.

In some embodiments, object detection based on low-resolution images from long-wave infrared sensors can achieve classification of detected objects by focusing the vehicle's computer vision processing resources on areas of interest related to the detected objects by adjusting control parameters of other sensors and/or deploying image processing resources.

For example, the integration time, aperture, filtering, or gain of a sensor (eg, a near infrared sensor or a visible light sensor) may be adjusted to enhance imaging of a region of interest associated with the captured object.

Additionally, the power level or field of view of an illuminator (e.g., a near infrared illuminator or a visible light illuminator) can be adjusted to enhance imaging of the region of interest. Computational control parameters can also be adjusted and applied to portions of the image associated with the region of interest.

The technology described in this patent application by Apple may provide improvements over existing computer vision systems for autonomous vehicles, such as increasing the effective range for detecting and classifying objects around the vehicle; more accurately classifying objects in low-light environments; and improving the safety of autonomous vehicle control systems and increasing the maximum safe driving speed in low-light environments.

The figure below is a block diagram of the multi-mode sensing system for object detection and identification of autonomous vehicles at night, which is shown in this patent by Apple, including the fields of view of various types of sensors installed on the vehicle. In the figure, 710 represents the autonomous vehicle; 720 represents the field of view of the visible light image sensor (VISIBLE FOV); 730 represents the field of view of the long-wave infrared sensor (LWIR FOV, with the largest field of view); 740 represents the field of view of the near-infrared image sensor (NIR FOV), which has a slightly smaller field of view but the largest effective detection distance.

Titan project came to an end? I believe this is because Apple does not compromise on high quality and high standards

Apple's patent application was filed in the third quarter of 2018, and it is not yet certain when its self-driving car will be available.

As early as 2014, Apple launched the Titan self-driving car project. Titan is a god who ruled the world in Greek mythology. Apple's original goal was to build a car that could subvert the industry, and the product was scheduled to be unveiled in 2019. But in 2016, Apple suddenly laid off a large number of employees on the project, and laid off employees again in early 2019. This series of actions made people wonder whether there was a problem with the self-driving project or with Apple.

According to foreign media analysis, one possible reason is the sluggish sales of Apple’s mobile phones in the market, and Apple has to cut spending on non-core programs.

In addition, Apple recently officially announced that it would abandon the AirPower wireless charging pad project. Apple officials said: "After a lot of effort, we confirmed that AirPower could not meet our high standards, so we canceled the project." The safety of self-driving cars is related to life, and Apple's product philosophy and high standards may not be able to keep up with the current design and manufacturing level. Therefore, Apple's uncompromising attitude towards product quality may also be a potential reason for the confusion of the Titan project.