NVIDIA has introduced a suite of new NVIDIA Omniverse™ libraries and NVIDIA Cosmos™ world foundation models (WFMs) designed to speed up the creation and deployment of advanced robotics solutions.
Powered by the latest NVIDIA RTX PRO™ Servers and NVIDIA DGX™ Cloud, these tools allow developers worldwide to build physically accurate digital twins, capture and replicate real-world environments in simulation, produce synthetic training data for physical AI, and create AI agents capable of understanding and interacting with the physical world.
“Computer graphics and AI are converging to fundamentally transform robotics,” said Rev Lebaredian, vice president of Omniverse and simulation technologies at NVIDIA. “By combining AI reasoning with scalable, physically accurate simulation, we’re enabling developers to build tomorrow’s robots and autonomous vehicles that will transform trillions of dollars in industries.”
Advancing Robotics Development with New NVIDIA Omniverse Libraries
The latest NVIDIA Omniverse software development kits (SDKs) and libraries are now available, enabling industrial AI and robotics developers to build and deploy next-generation simulation applications.
Key updates include:
-
Cross-Platform Robot Simulation: New SDKs support data interoperability between MuJoCo (MJCF) and Universal Scene Description (OpenUSD), giving more than 250,000 MJCF robot learning developers the ability to simulate across platforms seamlessly.
-
NuRec Neural Rendering: The Omniverse NuRec libraries and AI models introduce RTX ray-traced 3D Gaussian splatting, enabling developers to capture, reconstruct, and simulate real-world environments in 3D using sensor data.
-
Upgraded Simulation Tools: NVIDIA Isaac Sim™ 5.0 and open-source NVIDIA Isaac Lab 2.2 are now live on GitHub, featuring NuRec rendering and new OpenUSD-based robot and sensor schemas to help bridge the gap between simulation and reality.
Omniverse NuRec rendering has also been integrated into CARLA, a leading open-source simulator with over 150,000 users. Industry leaders including Foretellix, Voxel51, Boston Dynamics, Figure AI, Hexagon, RAI Institute, Lightwheel, and Skild AI are leveraging these tools to accelerate AI robotics development. Amazon Devices & Services is adopting them for a new manufacturing solution.
Also Read: Skild AI Unveils Skild Brain, a General AI Model for Robots
Cosmos World Foundation Models Push Boundaries of Synthetic Data Generation
With over 2 million downloads, NVIDIA’s Cosmos WFMs enable developers to produce large-scale, diverse datasets for training robotics systems using text, image, and video prompts.
New enhancements announced at SIGGRAPH include:
-
Cosmos Transfer-2: Coming soon, this update streamlines prompting and speeds up photorealistic synthetic data generation from ground-truth 3D scenes or spatial inputs such as depth, segmentation, and HD maps.
-
Ultra-Fast Model Distillation: A distilled Cosmos Transfer model reduces the traditional 70-step process to just one, enabling unprecedented performance on NVIDIA RTX PRO Servers.
Companies like Lightwheel, Moon Surgical, and Skild AI are already using Cosmos Transfer to rapidly simulate diverse training conditions for physical AI systems.
Cosmos Reason: Bringing Human-Like Understanding to Robots
NVIDIA has also unveiled Cosmos Reason, a new open, customizable, 7-billion-parameter reasoning vision language model (VLM) designed for physical AI and robotics. It allows robots and vision AI agents to reason with prior knowledge, physics understanding, and common sense enabling more accurate decision-making in real-world environments.
Applications include:
-
Automated Data Curation & Annotation: High-quality dataset preparation at scale.
-
Robotic Planning & Execution: Acting as a vision language action (VLA) model for complex, multi-step tasks.
-
Advanced Video Analytics: AI agents capable of large-scale video search, summarization, and root-cause analysis.
Uber, Magna, VAST Data, Milestone Systems, and Linker Vision are among the first adopters, applying Cosmos Reason to use cases ranging from autonomous delivery vehicles to smart city traffic monitoring and industrial visual inspections.
AI Infrastructure to Power the Next Generation of Robotics
To support these advancements, NVIDIA has launched AI infrastructure optimized for robotics workloads:
-
NVIDIA RTX PRO Blackwell Servers: A unified platform for training, simulation, synthetic data generation, and robot learning.
-
NVIDIA DGX Cloud on Microsoft Azure Marketplace: A fully managed cloud service for streaming OpenUSD- and RTX-based applications at scale, already adopted by Accenture and Hexagon.
Strengthening the Robotics Developer Ecosystem
NVIDIA is also investing in talent development and open collaboration, introducing:
-
OpenUSD Curriculum and Certification: Created with the support of AOUSD members such as Adobe, Amazon Robotics, Autodesk, Pixar, Siemens, TCS, and others.
-
Open-Source Collaboration with Lightwheel: Integrating policy training and evaluation frameworks into NVIDIA Isaac Lab for advanced reinforcement learning and robot manipulation tasks.
