Open source! Xingdong Jiyuan, Tsinghua University and Shanghai Qizhi Research Institute have launched a new trend in algorithms!

Currently, humanoid robots , as the highest embodiment of embodied intelligence, have attracted widespread attention from all walks of life. However, due to the high complexity of the structure of humanoid robots, their reinforcement learning training and the process of migrating from simulated environments to the real world are more challenging than those of quadruped robots.

Recently, the open-source humanoid robot end-to-end reinforcement learning training framework Humanoid-Gym, developed by embodied intelligence and humanoid robot company Star Era, Tsinghua University, and Shanghai Institute of Intellectual Property, was officially released. The system is fully open source, aiming to lower the threshold for the development of humanoid robot algorithms and help researchers in academia and industry around the world to innovate in the field of humanoid robots.

Reinforcement learning training of humanoid robots has always been a major challenge.

First, the humanoid robot has a highly complex body structure and a high degree of freedom in its joints, which makes the optimization space dimension of the control strategy amazing and the algorithm difficult to solve efficiently. Secondly, there are obvious modal differences between the training environment and the real environment of the robot. The trained strategy is difficult to transfer directly to the real world, and it requires a lot of manpower and material resources to debug repeatedly. Furthermore, there is no unified benchmark evaluation system so far, and it is difficult to weigh the pros and cons of the trained strategies, and it is also difficult to learn from and share the research results of predecessors.

It is based on the above background that the Humanoid-Gym framework was born.

The Humanoid-Gym framework significantly simplifies the training of humanoid robots and the difficulty of achieving sim-to-real conversion through its carefully designed reward function and domain randomization technology.

In addition to sim-to-real verification, another common practice is to use a second higher-precision simulation environment for preliminary verification (sim-to-sim). After Humanoid-Gym is open sourced, users can easily use the sim-to-sim conversion function through the framework to perform preliminary verification and screening in the higher-precision simulation environment Mujoco, thereby improving the efficiency and success rate of sim-to-real conversion.

Currently, the framework has been successfully verified on two robots under Star Era - Xiaoxing (1.2 meters high) and Xiaoxing max (1.65 meters high).

Humanoid-Gym marks StarMotion's positive attitude in promoting and sharing humanoid robot technology. By lowering the entry threshold, StarMotion hopes to inspire more innovative thinking and solutions to jointly promote the future development of humanoid robot technology. However, for now, from algorithm theory to engineering practice, this field still has many bottlenecks to be solved, and it urgently needs the continuous cohesion of global wisdom.

The release of Humanoid-Gym is the beginning of a new era of sim-to-real, but it is by no means the end. We look forward to global academic and industry colleagues joining this new wave and working together to promote the development of embodied intelligence technology!