DeepMind leads the innovation of robotics technology: from "robot constitution" to efficient decision-making

With the popularity in life and work, the coordinated and safe coexistence of robots and humans has attracted more and more attention. On January 4, 2024, Google DeepMind's robotics team announced three new advances, saying that these advances will help robots make faster, better and safer decisions. This update introduced three systems: AutoRT, SARA-RT and RT-Trajectory, which improved the robot's data collection, speed and generalization respectively. Among them, the more eye-catching is that DeepMind drafted a "Robot Constitution" and introduced it into the data collection system AutoRT to ensure that robot assistants do not harm humans.

The Robot Constitution, inspired by the Three Laws of Robotics proposed by American science fiction writer Isaac Asimov, is described as a set of "safety-focused cues" that instruct LLMs to avoid tasks involving humans, animals, sharp objects and even electrical appliances.

Robot Constitution

AutoRT is a data collection system that uses a visual language model (VLM) and a large language model (LLM) to work together to understand the environment, adapt to unfamiliar environments, and decide on appropriate tasks.

The system has safety guardrails, one of which is providing its decision-makers with a “robot constitution” — a set of safety-focused prompts to follow when choosing tasks for the robot.

To improve safety, DeepMind engineered the robot, set limits on its joints so that it stops automatically when the force on the joints exceeds a certain threshold, and installed a physical kill switch that operators can use to shut down the robot.

Google said it has deployed a fleet of 53 AutoRT robots in four different office buildings over the past seven months and conducted more than 77,000 trials.

Some robots are remotely controlled by human operators, while others operate according to scripts or fully autonomously using Google's learning model Robocs Transformer 2 (RT-2).

The robot used in the trial was equipped with only a camera, arm and mobile base.

“For each robot, the system uses the VLM to learn about its environment and the objects in its line of sight,” the report reads. “The LLM then proposes a list of creative tasks that the robot can perform, such as ‘put a snack on the counter,’ and plays the role of a decision maker, selecting the appropriate task for the robot.”

In addition, other new technologies from DeepMind include the SARA-RT system, an architecture designed to make the existing RT-2 model more accurate and faster.

The SARA-RT-2 model, after receiving a shorter history of images, was 10.6% more accurate and 14% faster than the RT-2 model.

The team also announced the RT-Trajectory system, which adds 2D contours to help robots better perform specific physical tasks, like wiping a table.

Reviewing Editor: Huang Fei