Artificial intelligence (AI) imagination to improve the safety of self-driving cars

Imagine an orange cat, then imagine that cat is coal grey, and now imagine that cat striding along the Great Wall. When you imagine this, a series of neurons in your brain fire quickly, and based on your previous understanding of the world, you come up with all sorts of images. In other words, it’s easy for humans to imagine an object with different properties. But while deep neural networks can perform as well as or better than humans on certain tasks, computers are still struggling with their “imagination” skills.

According to foreign media reports, a research team at the University of Southern California has developed an artificial intelligence (AI) technology that can use human-like abilities to imagine objects with different attributes that have never been seen before. The team consists of Professor Laurent Itti, doctoral students Yunhao Ge, Sami Abu-El-Haija and Gan Xin. The researchers said: "We are inspired by the generalization ability of human vision and try to simulate human imagination in machines. Humans can separate what they have learned based on attributes such as shape, posture, position, color, and then combine this knowledge to imagine a new object.

Inspiration for new AI systems (Image credit: University of Southern California)

The generalization problem in AI

For example, let's say you want to create an AI system that can generate pictures of cars. Ideally, you would be able to feed the algorithm some pictures of cars, and it would be able to generate a variety of cars, such as Porsches, Pontiacs, pickup trucks, etc., in various colors and at different angles.

This is one of the long-sought goals of AI, to create models that can extrapolate, meaning that given a few examples, the model can extract the underlying rules and apply them to a variety of novel, previously unseen examples. However, machines are usually trained only on sample features such as pixels, without taking into account the properties of objects.

The science of imagination

In this new study, the researchers used a concept called disentanglement to try to overcome this limitation. For example, disentanglement can be used to generate deepfakes, separating a person's facial movements from their identity, and then synthesizing new images and videos to replace the original person with another person, but with the original facial movements, similar to face-swapping technology.

The new method trains images and combines them (Image credit: University of Southern California)

Similar to the above method, unlike traditional algorithms that only use one sample, the new method uses a bunch of sample photos and mines the similarities between the two to achieve "controllable disentangled representation learning."

The method then reassembles the knowledge to achieve "controllable synthesis of new images," or what people call imagination. For example, in the case of Transformers, it can take the appearance of Megatron's car, but with the color and posture of Yellow Bumblebee and the background of Times Square in New York. The result is a Megatron car in the color of Yellow Bumblebee driving in Times Square, even though no one has seen such an example during training.

The method then reassembles the knowledge to achieve "controllable synthesis of new images," or what people call imagination. For example, in the case of Transformers, it can take the appearance of Megatron's car, but with the color and posture of Yellow Bumblebee and the background of Times Square in New York. The result is a Megatron car in the color of Yellow Bumblebee driving in Times Square, even though no one has seen such an example during training.

Understanding the world

While separation is not a new concept, the researchers say the framework is compatible with nearly any type of data or knowledge, broadening its application range. For example, separating race and gender-related knowledge by moving sensitive attributes out of the equation could enable fairer AI.

In medicine, the technology could help doctors and biologists discover more useful drugs by separating the function of a drug from its other properties and then recombining them to synthesize new medicines. Giving machines imagination could also help create safer AI, for example, by training self-driving cars to imagine and avoid dangerous scenarios they haven't seen before.