Table tennis AI robot beats humans! Flexible switching between forehand and backhand, can catch both net and high balls, professional coach: reached the level of intermediate players

Latest update time：2024-08-09

Reads：

Baijiao Xifeng comes from Aofei Temple
Quantum Bit | Public Account QbitAI

The table tennis team competition at the Paris Olympics is in full swing, and Google's robot has applied to compete.

The first robot agent that reaches human competitive level is released!

You see, without paying attention, you won a ball from the professional coach!

Quick switching between forehand and backhand, continuous attack is no problem~

Faced with some unexpected tactics, such as long balls, high balls and net balls, he can also respond calmly.

In actual tests, the robot adapted to the styles of different players in real time and eventually won all matches against beginners. It also had a 55% winning rate against intermediate players.

Little ping pong ball, take it!

American table tennis star Barney J. Reed, who played against it, gave it high praise: it exceeded expectations and has reached an intermediate level .

After watching its performance, netizens said: Can I buy it? I want it.

Even encounters can be handled calmly

Table tennis is a sport that requires high levels of physical strength, strategy, and skills, and it often takes humans years of training to master it.

Therefore, unlike pure strategic games such as chess and Go, table tennis has become an important benchmark for robots to test their comprehensive capabilities, such as high-speed movement, real-time precise control, strategic decision-making, system design, etc.

For example, when faced with different landing points of the ball, the robot needs to move quickly; when faced with an obvious out-of-bounds ball, the robot should choose not to catch it.

The team found 29 table tennis players of different skill levels to compete, including beginners, intermediate, advanced and above.

Humans and robots played three games, following standard table tennis rules. (However, since robots could not serve, humans served throughout the game.)

There have actually been corresponding studies on table tennis robots before. What’s special about this Google robot is that it can engage in comprehensive competitive matches with humans it has never seen before.

It can quickly adapt to various human playing styles.

For example, look at this player. At the beginning of the game, the robot was obviously still in the process of adapting, and the human defeated the robot with a big score of 9 to 2.

But after the next round, the robot was obviously familiar with the opponent's style and kept following the score closely.

Ultimately, the bot won all of its beginner matches against all its opponents, and had a win rate of 55 percent against intermediate players.

Although the robot currently has no way to defeat advanced players, from various feedback from humans, we can see that everyone is happy to play with this robot.

How to catch a small ping-pong ball?

Before introducing the method, let’s take a look at the hardware configuration of the table tennis robot.

The main body uses a 6-degree-of-freedom Swiss company ABB 1100 robot arm, which is installed on two Festo linear guides to enable it to move in a plane. The lateral moving guide is 4 meters long and the longitudinal moving guide is 2 meters long.

The robotic arm is equipped with a 3D printed racket handle and a racket covered with short-grain rubber.

How did such a little boy like Dengxi learn to play table tennis?

In summary, a hybrid training method combining reinforcement learning and imitation learning was used .

The team designed a hierarchical and modular strategy architecture, where the agent consists of a low-level skill library (LLC) and a high-level controller (HLC) .

LLCs are a set of specialized policies, each trained to perform a specific table tennis skill , such as forehand, backhand, serve, etc. These LLCs are trained using an evolutionary strategy algorithm in a simulation environment using a CNN architecture.

The training process uses a ball state dataset collected from the real world to ensure the consistency between the simulated environment and the real environment.

The HLC is responsible for selecting the most appropriate LLC for each ball .

It contains multiple components: a style strategy for selecting forehand or backhand; a rotation classifier for identifying the rotation type of the incoming ball; an LLC skill descriptor that describes the ability of each LLC; and a set of heuristic strategies for shortlisting candidate LLCs based on the current situation.

HLC also uses online learned LLC preferences to adapt to the opponent's characteristics and bridge the simulation-to-reality gap.

Specifically, the team first collected a small amount of human competition data, set the initial task conditions, then used reinforcement learning to train an agent in a simulated environment, and then deployed the strategy zero-sample to the real world.

The MuJoCo physics engine is used to accurately simulate the dynamics of the ball and the robot, including air resistance, the Magnus effect, etc. It also designs and handles topspin "correction" by switching different racket parameters in the simulation to simulate real-world topspin and backspin effects.

In the process of continuous competition between the Agent and humans, more training task conditions can be generated and training-deployment can be repeated.

As the robot's skills improve, the competition becomes more complex, but still based on real-world task conditions. As the robot collects data, it can also find gaps in its capabilities and then make up for them through continued training in a simulated environment.

In this way, the robot's skills can be automatically improved through iteration in a cyclic process that combines simulation and reality.

In addition, the robot can track the opponent's behavior and playing style to adapt to different opponents, such as which side of the table the opponent tends to hit the ball back.

This allows you to try different techniques, monitor your success rate, and adjust your strategy in real time.

In experiments with humans, the team also discovered that the robot has a weakness: it is not good at handling backspin balls.

The robot's chance of hitting the table was plotted against its estimate of the ball's spin, and its chance of hitting the table dropped significantly when faced with more backspin balls.

The researchers said the robot had difficulty handling balls that arced low and close to the table without hitting the table, and was limited in determining the ball's spin type in real time.

This is not the first time Google has created a table tennis robot

Google started researching robots playing table tennis a long time ago. The team has a lot of other related research:

For example, in Google’s previous i-Sim2Real research, the trained robot could play basketball against humans for up to 340 consecutive times without dropping the ball, which is equivalent to playing continuously for more than 4 minutes.

Other teams have also had table tennis robots, such as this one, which can also serve:

Teams like the Japanese national team and the Chinese Taiwanese team also use robots to train their own Olympic athletes.

Some friends may wonder, what is the difference between this robot and the one released by Google this time?

Some netizens gave an explanation:

Google's take on this is AI Agents that work through video input rather than pre-programmed algorithms.

So, when can we see them play against our national team? (Doge)

Project homepage:
https://sites.google.com/view/competitive-robot-table-tennis/home?utm_source&utm_medium&utm_campaign&utm_content&pli=1

Reference links:
[1]https://x.com/GoogleDeepMind/status/1821562365931855970
[2]https://x.com/arankomatsuzaki/status/1821360354653344190
[3]https://x.com/lgraesser3/status/ 1547942995139301376
[4]https://www.reddit.com/r/singularity/comments/1en8vrg/google_deepminds_aipowered_robot_plays_table/
[5]https://www.youtube.com/watch?v=u3L8vGMDYD8