Robot simulation technology that cannot be ignored

In recent years, there have been endless cases of AI-empowered robots. However, it is also worth noting that as the amount of data increases exponentially, it also brings considerable challenges to robot simulation technology.

Nvidia officially launched the robot platform Isaac in 2018, which includes a complete set of robot simulators called Isaac SIM, which can build and test virtual robots in a variety of realistic environments. With Isaac Sim, robotics experts can build realistic environments that validate a robot's physical design and fully train the robot's software stack to ensure performance. Additionally, synthetic data sets can be built during the simulation and used to train AI models in the robot perception system. Researchers can use reinforcement learning to train models in the robot control stack.

Every year since then, Isaac SIM has been continuously improved to expand support for robot simulation technology. Recently, NVIDIA announced a number of important updates to Isaac SIM at CES 2023.

These include improved performance and functionality for special use cases such as manufacturing and logistics robots. Adding people to simulated environments is now supported. More assets and popular robots are pre-integrated to reduce pre-simulation time.

At the same time, for ROS developers, support for ROS2 Humble and Windows has been added. For robotics researchers, the new version has a host of new features designed to advance reinforcement learning, collaborative robot programming, and robot learning.

It is NVIDIA's continuous expansion on the Isaac platform that has allowed it to rapidly rewrite the robotics industry in a few years. From AMR to sensors, Isaac's ecosystem partners have spread across various industrial segments.

Integrate more scene optimization

In order to speed up the robot simulation process, NVIDIA has added support for many robot scenarios to speed up the robot development process. This update includes many popular scenarios such as warehousing and logistics, manufacturing, and fleets.

Warehouse logistics: Warehouse logistics is one of the most important areas of robotics innovation. Several tools in Isaac Sim cater to the needs of simulating warehouse robots. Previous versions included a repository builder. Isaac Sim now includes a tool for building realistic conveyor belts. There is also a personnel simulator that adds digital humans to the warehouse environment. This will help validate perception and safety systems for robots working near humans.

Manufacturing: Manufacturing environments have long used robotic robots to automate processes. The 2022.2 version has many new features in handling motion control of manipulator robots. The updated motion generation extension in Isaac Sim simplifies the integration and benchmarking of motion control algorithms in simulations. The included algorithm, RMPFlow, creates smooth trajectories for the robot, with intelligent collision avoidance. In addition to improved performance, this release offers a number of usability improvements, including a graphical editor for Lula robot description files.

Fleet Optimization: Introducing the cuOpt extension for Isaac Sim, a deployment of the NVIDIA cuOpt engine integrated directly with Isaac Sim. NVIDIA cuOpt is an operations research optimization API that helps developers create complex real-time fleet routing. These APIs can be used to solve complex routing problems with multiple constraints and provide new capabilities such as dynamic rerouting, job scheduling, and robot path planning while leveraging sub-second solver response times.

Other important upgrades

As an important annual update, Isaac SIM also released a number of other updates. These include:

ROS support upgrade: Support for ROS has been upgraded to support ROS 2 Humble, and the Isaac ROS, also based on the Humble version, can be easily simulated and tested in this version. Support for ROS 2 has also been added on Windows machines.

Research Tools: Important new features designed for robotics researchers include performance improvements for Isaac Gym (reinforcement learning). Isaac Cortex for collaborative robot programming has new examples. Finally, a new open source tool, Isaac ORBIT, provides simulated operating environments and benchmarks for robot learning and motion planning.

Sensor support: With NVIDIA RTX technology, Isaac Sim can now render physics-correct data from sensors in real time. When simulating lidar with RTX, ray tracing can provide more accurate sensor data under various lighting conditions or when reacting to reflective materials.

The ecosystem continues to expand

With the deepening of Isaac in various fields, more than one million developers and a thousand companies are now using the NVIDIA Isaac robot platform, covering multiple industries from logistics, manufacturing to retail, energy, sustainable agriculture, etc., NVIDIA Many cooperation cases have also been announced recently.

Telexistence deploys beverage replenishment robots in 300 convenience stores in Japan; to improve safety, Deutsche Bahn is training AI models to handle unexpected extreme events that are critical but rarely occur in the real world, such as dropped luggage On train tracks; Sarcos Robotics is developing robots that pick and place solar panels in renewable energy installations.

Festo uses Isaac Cortex to simplify programming of collaborative robots and transfer simulation skills to physical robots; Fraunhofer is using Isaac Sim’s physics-compliant, full-fidelity visualization capabilities to develop advanced AMR; Flexiv is using Isaac Replicator to generate data for training AI models Synthetic data.

While training robots is important, simulation also plays a key role in training human operators to work with and program robots. Ready Robotics is using Isaac Sim to teach programming for industrial robots; Universal Robotics is using Isaac Sim to develop its workforce and train terminal operators in the cloud.

Summarize

In the past five years, robots have gained widespread recognition in the market for their capabilities in automation, intelligence, and solving manpower shortages. According to ABI Research, the installed base of industrial and commercial robots will grow from 3.1 million units in 2020 to 20 million units in 2030, an increase of more than 6.4 times.

In order to develop, validate and deploy these new AI robots, companies need to use simulation technology to put these robots into real-life scenarios. Isaac SIM will help robot simulation technology continue to move forward through innovation again and again.