AI is moving towards a true “intelligent learning body”: it can look back to the past and solve complex tasks

　　The British journal Nature published an artificial intelligence research result: an American team reported a type of reinforcement learning that can look back to the past and solve complex tasks, which truly improved the way of exploring complex environments and is expected to be applied in the fields of robotics, language understanding and drug design. This type of algorithm is collectively called "Go-Explore", and it has already scored higher than human players and advanced artificial intelligence systems in an algorithmic challenge of a classic game. This achievement is considered to be an important step towards realizing a true "intelligent learning body".

　　Reinforcement learning allows artificial intelligence systems to make decisions by exploring and understanding complex environments, and learn how to obtain rewards in the best way. Rewards can include robots reaching a certain location or reaching a certain level in a computer game. However, when encountering complex environments with little feedback, current reinforcement learning algorithms can easily run into obstacles, which is very distressing for artificial intelligence experts.

　　OpenAI is an artificial intelligence non-profit organization jointly established by many Silicon Valley giants. Its promoters include Sam Altman, president of the American startup incubator Y Combinator, and Elon Musk, founder of the American Space Technology Exploration Company (SpaceX). Its goal is to prevent the catastrophic impact of artificial intelligence and promote the positive role of artificial intelligence. This time, OpenAI scientists Edran Ekfet, Just Huizinga and their team proposed two major obstacles to effective exploration and designed a class of algorithms to solve these obstacles.

　　The researchers said that "Go-Explore" can fully explore the environment while building an archive to remember where it has been, ensuring that it does not forget the route to a promising mid-term stage or final victory (reward). Its scores in Atari classic games exceeded those of human players and advanced artificial intelligence systems. The researchers used this type of algorithm to solve 2,600 Atari games that had not been solved before, verifying the potential of this type of algorithm. "Go-Explore" scored four times as much in the algorithm challenge "Montezuma's Revenge" as before, and also scored higher than the average level of human players in another algorithm challenge "Maya's Adventure". In contrast, previous algorithms did not get any points.

　　The "Go-Explore" algorithm was also able to complete a simulated robotics task in which it had to use a robotic arm to pick up an object and place it on one of four shelves, two of which were behind two doors.

　　The researchers note that the simple principle of remembering and returning to promising areas of exploration is a powerful, general approach to exploration, and they believe their new algorithm has potential applications in robotics, language understanding, and drug design.