Jim Fan Outlook: Robotics is about to see a GPT-3-style breakthrough

Scientists On September 19, the media The Decoder published an interesting report that Nvidia senior scientist Jim Fan recently predicted that technology will usher in a revolutionary breakthrough similar to GPT-3 in the field of language processing in the next two to three years. He called it the "GPT-3 moment" in the field of robotics.

Jim Fan, a scientist who received his Ph.D. from Professor Fei-Fei Li at the Stanford University Vision Lab, has conducted research in a variety of cutting-edge fields, including multimodal basic models, reinforcement learning, and more. His internships at top institutions such as Google Cloud, OpenAI, and Baidu Silicon Valley Lab have given him a wealth of research experience. Currently, he leads NVIDIA's exploration of AI, especially focusing on the "Project Groot" project, which aims to build a basic model for humanoid robots.

In an interview with Sequoia Capital, Dr. Fan expressed an optimistic outlook on the future of robotics. He believes that although it will take some time for robots to be fully integrated into daily life, research on basic robot models will make significant progress in the next two to three years. This breakthrough will be similar to the shocking impact of GPT-3 in the field of language processing, opening a new chapter for robotics.

Dr. Fan further pointed out that the introduction of robots into daily life is not only a technical challenge, but also involves many factors such as cost, production scale, safety, privacy protection and regulatory compliance. He emphasized that the world is designed around the human form, so humanoid robots have a natural advantage in performing various tasks. He predicted that as the technology matures, humanoid robots will be able to perform various tasks that humans can do, and the hardware ecosystem that supports this vision will be ready in the next few years.

NVIDIA has adopted an innovative strategy to promote the development of robot AI, combining three types of data sources: Internet data, data, and real-world robot data. Dr. Fan believes that this diversified data fusion approach is the key to success and can make up for the shortcomings of a single data source. In addition, NVIDIA has also developed technologies such as "Eureka", which uses language models to automatically generate reward functions for robot training, thereby accelerating the automation process.

In addition to real-world applications, Dr. Fan's team is also exploring AI agent technology in virtual environments, such as games. They found commonalities between these fields and are committed to developing a unified model that can control both virtual and physical agents. This research not only helps improve the adaptability of robots in complex environments, but also lays a solid foundation for the widespread application of robotics in the future.