New NVIDIA Neural Graphics SDK Enables Everyone to Create Metaverse Content

More than a dozen tools and programs, including the newly released NeuralVDB and Kaolin Wisp, enable millions of designers and creators to easily and quickly create 3D content.

Creating 3D objects for games, virtual worlds (including the metaverse), product design, or visual effects scenes has always been a highly detailed task, and even the most skilled artists must balance the fidelity of detail with deadline and budget pressures.

It takes a long time to create objects that look the same in the physical world, and it is even more complicated when multiple objects and characters need to interact in the virtual world. Simulating physics has become as important as simulating light. For example, robots in virtual factories must not only look exactly the same as real robots, but also have exactly the same load-bearing and braking capabilities.

It’s hard to do this perfectly, but it’s a huge opportunity. In addition to product design, it affects multiple trillion-dollar industries such as transportation, healthcare, telecommunications, entertainment, etc. Eventually, the amount of content created in the virtual world will exceed the content created in the physical world.

To simplify and shorten that process, NVIDIA today announced new research and tools that apply neural graphics to create and animate 3D objects and worlds.

These SDKs include NeuralVDB, a breakthrough update to the industry standard OpenVDB, and Kaolin Wisp, a Pytorch library that builds a neural field research framework. They not only simplify the creative process for designers, but also enable millions of non-professional designers to easily create 3D content.

Neural graphics is a new field that combines AI and graphics to create accelerated graphics pipelines that learn from data. The addition of AI improves results, helps automate design choices, and opens up new opportunities for artists and creators that are currently unimaginable. Neural graphics will redefine the way users create, simulate, and experience virtual worlds.

These SDKs and research will advance every stage of the content creation process, including:

3D content creation

●Kaolin Wisp: Complementing Kaolin, a PyTorch library that reduces the time to test and implement new technologies from weeks to days, making 3D deep learning research faster, Kaolin Wisp is a library for neural research that builds a set of common tools and frameworks to accelerate new neural research.

●Instant Neural Graphics Primitives: A new way to capture the shapes of real-world objects and the inspiration behind NVIDIA Instant NeRF, a reversible rendering model that turns still image sets into digital 3D scenes. The technology and associated GitHub code speed up the process by up to 1,000 times.

●3D MoMa: A new reversible rendering pipeline that allows users to quickly import 2D objects into a graphics engine and create 3D objects that can be modified with realistic materials, lighting, and physics.

●GauGAN360: An evolution of NVIDIA’s GauGAN. This AI model can turn crude scribbles into realistic paintings. GauGAN360 generates 8K 360-degree panoramas that can be ported into Omniverse scenes.

●Omniverse Avatar Cloud Engine (ACE): This new cloud API, microservices and toolset enables the creation, customization and deployment of digital human applications. Built on NVIDIA’s unified computing framework, ACE enables developers to seamlessly integrate NVIDIA’s core AI technologies into their avatar applications.

Physics and animation

●NeuralVDB: A breakthrough improvement to OpenVDB, the current industry standard for volumetric data storage. NeuralVDB uses machine learning to introduce a compact neural representation that significantly reduces memory usage to support higher resolution 3D data.

●Omniverse Audio2Face: An AI technology that generates vivid facial animations from audio sources. It is ideal for interactive real-time applications and as a traditional facial animation creation tool.

●ASE: Animation Skill Embedding: A method for making physically simulated characters act in unfamiliar scenarios in a more responsive and realistic manner. It uses deep learning to teach characters how to respond to new tasks and actions.

●TAO Toolkit: A framework that enables users to create accurate, high-performance pose estimation models that can use computer vision to estimate what a person can do in a scene much faster than current methods.

Experience

●Image feature eye tracking: A research model that relates pixel rendering quality to user reaction time. It enables fast-paced interactive computer graphics applications (such as competitive games) to perform better by predicting the best combination of rendering quality, display properties, and browsing conditions to achieve minimal latency.

●VR Holographic Glasses: New VR glasses design developed with Stanford University delivers full-color 3D holographic images in a groundbreaking 2.5 mm thick optical stack.

Join NVIDIA at SIGGRAPH to learn more about the latest research and breakthroughs in graphics, AI, and virtual worlds. See the latest innovations from NVIDIA Research and get access to the full suite of NVIDIA SDKs, tools, and libraries.