OpenAI spinoff releases AI model: a major step forward in robot learning tasks

March 13, 2024 — In a major breakthrough, OpenAI spinoff Covariant announced the success of building a new artificial intelligence model that enables robots to learn to perform a variety of tasks like humans. This development marks a step towards more flexible and autonomous robotics.

In the summer of 2021, OpenAI shut down its robotics team, citing a lack of training data as a hindrance to progress. However, three early research scientists at Covariant found a solution in their startup, founded in 2017. They used years of data from fleets of picking robots in warehouses around the world, as well as text and videos from the internet, to create a new model called RFM-1. This model combines the reasoning power of large language models with the physical dexterity of advanced robots.

The RFM-1's capabilities are impressive. Users can direct the model to perform tasks using five different types of input: text, images, videos, robot instructions, and measurements. For example, a user can show the model an image of a bin full of sports equipment and tell it to pick up a bag of tennis balls. The robot will then perform the task and adjust as needed.

Although the model has "human-like" reasoning capabilities, there are still limitations. During the demonstration, it was found that when some new concepts were proposed, the model might not fully understand them. This shows that the model needs more training data and further improvement.

Covariant said they plan to launch the model to customers in the coming months and hope to continuously improve its performance and efficiency in real-world environments. They deployed the model in environments such as warehouses for testing and interacted with real-world instructions, objects, and environments.

As AI technology advances, competition among companies using it to power robotic systems is heating up. While there are still some kinks to work out, Covariant is committed to continually learning and improving their models so that robots can better adapt to changing environments.

In the near future, one can expect to see wider applications of robotics in various fields, and the ability of robots to learn and perform tasks will continue to improve as more data and improvements are made available.