End-to-end pure vision! OpenAI's humanoid robot performs to celebrate New Year

The house is white and comes from the Aofei Temple
Qubit | Official account QbitAI

In celebration of the Chinese New Year, the humanoid robot invested by OpenAI greets the new year through the latest video (doge)

Two people came up and performed a show of taking a package from the shelf, turning around and putting it into the basket:

The camera continued to advance, and the two robots leaning against the wall were constantly lifting, dropping, and lifting Tetris blocks.

On the other side, the charged robot gently unplugged the plug, squatted slightly, and placed the plug in the designated position.

There was also a guy coming to charge next to him:

In a small room, a smiling robot moved forward slowly, then smoothly picked up the dolls on the ground and put them back into the toy basket.

Finally, the camera follows a tour guide robot as it patrols the venue. The robots just introduced are still performing their duties and doing their own things.

Officials stated that this video is from the beginning to the end——

Every action taken by the robots is controlled end-to-end based on the visual neural network .

And the video is played at its original speed, without acceleration, editing, or remote control.

Some netizens are quite satisfied with the anthropomorphic simulation effect:

No, no, no, you're not still waiting for someone to unzip the zipper on the back of the robot, then take off the helmet and get off the balance car!

Wondering how this is done?

The official slapped out a piece of information and revealed the principle.

come on show

In order to fully develop and utilize its own humanoid robots, the company behind it, 1X Technologies (referred to as 1X) , has been pursuing the most common method since last year, which is to use visual neural networks to allow robots to learn movement behaviors end-to-end .

In the official video released, the behavior of each robot is controlled by a visual neural network with a frequency of 10Hz.

It mainly controls the robot's movement, arms, torso and head.

This video contains no remote operations, no computer graphics, no editing, no video acceleration, and no scripted track playback.
All this is controlled through neural networks and is autonomous.

According to reports, 30 robots appeared in this display video (currently, the company has a total of 50 robots of this type)

To train machine learning models that generate behaviors demonstrated in videos, the 1X team assembled a high-quality, diverse demonstration dataset from these 30 robots .

This data is used to train a basic model that can understand a wide range of physical behaviors, such as cleaning a house, picking up items, and socializing with humans or other robots.

The team then fine-tuned the basic model into a more specific set of functions, such as a model for general door operations or a model for warehouse tasks.

Then, fine-tune the fine-tuned model so that the behavior it generates is consistent with a specific task (such as opening a specific door) .

This training method allows the team to collect and train data on a desktop GPU in a few minutes, allowing the model to master new skills.

More importantly, in this way, the robot can learn short-term mobile operation skills without being limited by the number of AI engineers.

The robot on display this time is the first-generation product of 1X Company, named EVE . It is specially designed for the work environment and is mainly used in industry, logistics, retail and security fields.

EVE is humanoid from head to legs, and then uses two wheels instead of human feet.

It is 1.86 meters tall, weighs 86 kilograms, has a top speed of 14.4 kilometers per hour, a carrying capacity of 15 kilograms, and a battery life of 6 hours.

Although it has not been directly announced, due to the high overlap between design and business, EVE is regarded by the industry as the benchmark for Musk's Tesla robot Optimus.

In May last year, Firstpost news showed that EVE defeated Optimus in a humanoid robot competition.

Bernt Bornich, CEO and founder of 1X, said that currently, EVE is already on the job and has been put into commercial application in some parts of Europe and the United States. At least two industrial sites have deployed EVE for security work.

At the beginning of this year, 1X also appeared at a train station in Oslo, the capital of Norway, pouring coffee for passengers.

Netizens quite approve of EVE’s flexibility:

But some people have also raised some questions that have yet to be solved and thought about:

The first is stability, the second is versatility, and the third is the need for huge data for training.

Who is the company behind it?

Behind EVE is a Norwegian company 1X , founded in 2014 and formerly known as Halodi Robotics.

The company focuses on creating humanoid robots capable of human-like movements and behaviors and deploying them in real-world scenarios.

In particular, the current product leader of 1X is Eric Jang .

Eric Jang graduated from Brown University with a major in computer science. After graduation, he worked at Google X for 6 years as a senior research scientist in robotics.

In April 2022, Eric joined 1X to lead the AI ​​department.

Since 2022, 1X has been cooperating with OpenAI on robot AI models; currently, some functions of EVE are supported by ChatGPT.

In March last year, 1X officially had the "backing" of OpenAI - it received US$23.5 million in financing led by the OpenAI Venture Fund, with a post-investment valuation of approximately US$100-200 million .

This is also the first hardware company OpenAI has invested in.

Brad Lightcap, COO of OpenAI, once publicly confessed: "The timing for investing in 1X is when its robotic hardware has matured and can open up a wider labor market with the support of AI."

However, Eric Jang also said that 1X is not exclusively bound to OpenAI .

We are still testing other companies’ large models, and then we are also making some small models of actions and tasks ourselves.

1X, which announced two rounds of financing last year, officially announced last month that it had completed a $100 million Series B round of financing.

This money will not only be used to expand the data collection strategy of embodied intelligence, but also to strengthen the marketing promotion of NEO robots.

That’s right! In addition to EVE, 1X is also working overtime to develop the second-generation robot NEO (named after the protagonist of "The Matrix") .

Unlike EVE's wheeled humanoids, NEO is a pure humanoid robot with 55 degrees of freedom on its feet and can walk freely.

After having legs, NEO can be said to be completely designed according to human appearance and movement methods:

Has a head, body, arms and legs, can walk, grasp objects, and communicate through facial expressions.

At the same time, it uses high-torque joints, which compared with traditional high-performance gear power, has been comprehensively enhanced in terms of control, compatibility, power output, toughness, and maintenance.

Currently, NEO is still in the research and development stage, but the official website claims that this model can already walk through doors and climb stairs naturally and accurately.

The official statement of 1X is that it plans to officially release NEO this year and carry out mass production.

I wonder if the 1X humanoid robot with two legs can still be far ahead of Musk's Optimus?

Waiting to watch the show.jpg

—Contact the author— ​

Click here ???? Follow me and remember to star~