Apple's new Project Titan patent: Detecting hidden objects in the vehicle's field of view

According to foreign media reports, the US Patent & Trademark Office has published a patent application from Apple, which involves a LWIR (long wave infrared) foveal vision system for detecting objects with low visibility in a vehicle's wide field of view.

(Image source: www.patentlyapple.com)

Apple said that it is difficult for vehicles to completely avoid objects on the road, especially moving objects such as animals that cross the vehicle's path. Although autonomous vehicles or semi-autonomous vehicles are equipped with various sensor systems for object detection to avoid such objects. However, in poor lighting conditions, including nighttime, low-visibility weather (such as fog, snow, rain, etc.), glare, and low visibility of objects, traditional sensor systems often fail.

For example, monochrome sensors typically require active illumination to detect objects in low-light conditions and tend to saturate in the presence of glare. As a result, objects are not easily detected by monochrome sensors in low-light conditions and in the presence of glare (due to external light sources, such as the headlights of other vehicles).

Other traditional sensor systems do not require active illumination by using passive sensors, such as long-wave infrared sensors. However, such sensor systems are often unable to identify objects when lighting conditions are poor and object resolution is low. Many other traditional sensor systems cannot be deployed in vehicles for object detection due to cost, weight or size constraints. Therefore, in poor lighting conditions, the challenge of avoiding objects increases if they are hidden from the field of view of traditional sensor systems.

Apple's patented technology can help cars drive autonomously in adverse lighting conditions. Different night environments have different levels of ambient light, which can affect the sensitivity of the vehicle's sensor suite. Urban environments usually have ample ambient light, coming from street lights, neighboring buildings, etc. However, the ambient light in rural environments is very limited, mainly coming from starlight, moonlight, and atmospheric light; suburban areas have ambient light from street lights, houses, and other vehicles.

In such lighting conditions, objects may be hidden from the vehicle’s field of view. For example, in the dark, a deer may be hidden beside the street, undetectable by the sensor, but quickly cross the street when the vehicle approaches. Because these objects have a thermal signature, long-wave infrared (LWIR) vision systems can detect objects at different distances in adverse lighting conditions.

However, LWIR has a narrow field of view and low resolution, so it may not be able to detect hidden objects. Apple's technology focuses the resolution of the LWIR vision system to a specified area in the field of view to detect and identify hidden objects. Using this LWIR foveal vision technology, the heat energy of hidden objects can be detected with higher resolution in a specified area of ​​a wide field of view. In addition, stereo vision can be used to create an extended depth of field of view using multiple LWIR images to obtain more details of hidden objects in a specified area. In addition, the LWIR foveal vision system can also be used in combination with other imaging or detection systems (including monochrome sensors, RGB sensors, lidar sensors, etc.).

As shown below, Apple's patent figure FIG. 8 is an example diagram of the forward longitudinal far field of view of the LWIR foveal vision system; patent figure FIG. 14 is an example operation diagram for object detection; patent figure FIG. 5 is an example diagram of a sensor suite that maximizes the field of view while ensuring spatial resolution.

(Image source: www.patentlyapple.com)