NVIDIA releases major Omniverse upgrade supporting generative AI and OpenUSD

The latest platform updates, access to Adobe Firefly, the introduction of OpenUSD to RealityKit and the Ada architecture system will jointly accelerate various interoperable 3D workflows and industrial digitalization processes

LOS ANGELES - SIGGRAPH - August 8, 2023, Pacific Time - NVIDIA today released an important version update of the NVIDIA Omniverse™ platform, providing developers and industrial enterprises with new basic applications and services to help them use the OpenUSD framework and generative AI to Optimize and enhance its 3D workflow.

Omniverse is an OpenUSD native software platform that enables connection, description and simulation across 3D tools and applications. This update will speed up the creation of virtual worlds and advanced workflows, driving industrial digitalization. Cesium, Convai, Move AI, SideFX Houdini and Wonder Dynamics are now connected to Omniverse via OpenUSD.

This platform update mainly upgrades Omniverse Kit (the engine used to develop native OpenUSD applications and extensions), as well as NVIDIA Omniverse Audio2Face™ basic applications and spatial computing functions.

Rev Lebaredian, vice president of Omniverse and Simulation Technologies at NVIDIA, said: "Industrial companies are racing to digitize their workflows, which is increasing the need for an OpenUSD-powered, connected, interoperable 3D software ecosystem. The latest Omniverse update allows developers to With OpenUSD, use generative AI to enhance your tools and enable enterprises to build larger, more complex world-class simulations as digital testing grounds for their industrial applications.”

Important improvements

Updates to Omniverse Kit include:

● New modular app building: The new Omniverse Kit Extension Registry is a central repository for accessing, sharing and managing Omniverse extensions, making it easier than ever for developers to turn features on or off in their apps. NVIDIA provides more than 600 core Omniverse extensions to build custom applications.

● New developer templates and resources: New application and experience templates allow developers who are new to using OpenUSD and Omniverse to get started easily with minimal coding work.

● Improving efficiency and user experience: New rendering optimization features take full advantage of improvements in the NVIDIA Ada Lovelace architecture in NVIDIA RTX™ GPUs and fully integrate DLSS 3 technology into the Omniverse RTX renderer. The new AI denoiser enables real-time 4K path tracking of large industrial scenes.

● Spatial integration driven by native RTX: New extended reality (XR) developer tools allow users to build native spatial computing options in Omniverse-based applications, allowing users to flexibly experience 3D projects and virtual worlds according to their own ideas.

These platform updates are reflected in the Omniverse base application. Omniverse base apps are fully customizable reference apps that creators, businesses, and developers can copy, extend, or improve. Upgraded applications include:

● Omniverse USD Composer: enables 3D users to assemble large OpenUSD-based scenes

● Omniverse Audio2Face: Provides access to a generative AI application programming interface that creates realistic facial animations and gestures simply from audio files. Multilingual support and a new female base model have now also been added.

OpenUSD expands Omniverse ecosystem

NVIDIA has also released a series of frameworks, resources and services for developers and enterprises to accelerate the adoption of Universal Scenario Description (i.e. OpenUSD).

Additionally, NVIDIA announced the construction of a new Omniverse Cloud API for developers to more seamlessly implement and deploy OpenUSD workflows and applications. For example, ChatUSD is a large language model (LLM) copilot tool for developers that can answer USD knowledge questions or generate Python-USD code scripts.

The new Omniverse connector powered by OpenUSD is now available, bringing more possibilities for industrial companies to break down data silos in complex production pipelines.

Adobe and NVIDIA announced plans to provide Adobe Firefly (Adobe's creative generative AI model series) as an API of Omniverse to developers and creators to help them improve the design process. This move expands the cooperation between the two parties in Adobe Substance 3D, generative AI and OpenUSD projects.

Wonder Dynamics has connected to Omniverse and provides new OpenUSD export support through its AI platform Wonder Studio. The Wonder Studio AI platform automatically animates, adds lighting, and composites computer-generated characters into real-time action scenes. New OpenUSD export support will allow artists to generate and export complete 3D scenes using only video captured by a single camera.

Reality capture models produced by Luma AI in USDZ format can be imported into Omniverse at any time. Tools from avatar company Convai and character engine company Inworld AI are also connected to Omniverse. With AI tools such as Convai, creators can add characters to the digital twin environment, which can provide information about the environment and objects, or act as tour guides or virtual robots. Move AI implements single-camera motion capture through the Move One application, and the captured images can be used to generate 3D character animations, which are then exported to OpenUSD and used in Omniverse.

Omniverse users can now build content, experiences and applications that are compatible with other OpenUSD-based spatial computing platforms such as ARKit and RealityKit. Additionally, new support for the Khronos Group OpenXR open standard expands Omniverse’s reach to additional headsets from manufacturers such as HTC VIVE, Magic Leap and Varjo.

SideFX Houdini users can now also load Houdini Digital Assets directly into the Omniverse window, making interconnected Houdini-based workflows more seamless. Cesium's Omniverse extension Cesium for Omniverse supports 3D Tiles. 3D Tiles is an open standard for transporting large-scale geospatial data sets in virtual worlds, including those supported by OpenUSD. CGI.Backgrounds A variety of ultra high definition HDRi maps are now available in USD Composer. Cadence DataCenter Design Software™, now available on Omniverse, helps users view computational fluid dynamics simulations in a complete digital twin context. With Cadence Data Center Extensions, users can plan, test, and validate design and operational considerations before implementation. The Blackshark.AI world digital twin platform is now connected to Omniverse.

Customers go digital with Omniverse

Multiple customers are using Omniverse for tasks ranging from simulating robots to training AI models and improving animations.

The Boston Dynamics AI Institute is using Omniverse to simulate robots and their interactions to advance the design of new robots and control systems. Continental, a leader in the industrialization of automotive and autonomous systems, is using Omniverse in its mobile robotics business to generate physics-compliant synthetic data at scale to train computer vision AI models and perform system integration testing.

Volvo Cars has converted its digital twin to OpenUSD format and used Omniverse to create immersive visualizations to help customers make purchasing decisions online.

Brand design and experience agency Marks Design is using Omniverse and OpenUSD to simplify and make collaboration more efficient while improving its animation, visualization and rendering workflows.

New Omniverse system and partners

NVIDIA is collaborating with system manufacturers around the world to provide RTX workstations optimized for Omniverse to millions of designers, architects and engineers. These new systems can be equipped with up to four NVIDIA RTX 6000 Ada Generation GPUs and paired with NVIDIA Omniverse Enterprise software to accelerate OpenUSD world building, generative AI-powered collaborative design, and other industrial digital applications.

Omniverse users can also take full advantage of the performance of the new NVIDIA L40S GPU. Accelerate graphics-intensive workloads with this powerful, general-purpose data center GPU.

Availability

The latest version of Omniverse is now available as a free downloadable public beta and will be available through Omniverse Enterprise in the near future.

Learn more about NVIDIA Omniverse and watch NVIDIA founder and CEO Jensen Huang's SIGGRAPH keynote.