New project develops AI vision system for autonomous driving

According to foreign media reports, a team led by Professor Amit Roy-Chowdhury of the University of California, Riverside, received a grant of nearly $1 million from the Defense Advanced Research Projects Agency (DARPA) of the United States Department of Defense to study adversarial attacks on computer vision systems. The project is part of the Total Machine Vision Disruption project in the DARPA AI Explorations project, and the research results will be widely used in the fields of autonomous vehicles, surveillance, and defense. Team members include researchers Srikanth Krishnamurthy, Chengyu Song, and Salman Asif from the University of California, Riverside, and PARC, a research and development company under Xerox.

(Image credit: University of California, Riverside)

When people see an object, they also notice the entire scene around it. This wider visual context makes it easier for people to detect and interpret irregularities. A human driver notices a sticker on a stop sign, knows that the sticker does not change the meaning of the sign, and stops anyway. However, a self-driving car using a deep neural network for object recognition may not recognize the stop sign because of the sticker and have difficulty navigating the intersection.

No matter how good a trained computer algorithm is at identifying changes in a target, image perturbations always increase the likelihood that the computer will make the wrong decision or recommendation. The vulnerability of deep neural networks to image processing makes them a target for hackers intent on interfering with decisions and actions powered by visual AI.

"If an object is in an inappropriate location, the defense mechanism is triggered," Roy-Chowdhury said. "We can do this even if part of the image is perturbed, like a sticker on a stop sign." For example, when people see a horse or a boat, they also want to see certain objects around them, such as a barn or a lake. If one of these images is perturbed, such as a horse standing in a car dealership or a boat floating in the clouds, people can recognize the error. Roy-Chowdhury's team hopes to use this ability in computers.

To do this, researchers first need to identify the types of attacks that are possible. The DARPA project will focus on using information from the visual environment to generate adversarial attacks to better understand the weaknesses of machine vision systems. "We will perturb the image system so that the computer gives the wrong answer, which may help design defenses against attacks," Roy-Chowdhury said.