NVIDIA has launched a range of AI and simulation tools to support robotics developers accelerate work on AI-enabled robots, including humanoids. The announcement was made at the Conference for Robot Learning (CoRL) in Munich, Germany.

The new tools include general availability of the NVIDIA Isaac Lab robot learning framework; six new humanoid robot learning workflows for Project GR00T, an initiative to accelerate humanoid robot development; and new world-model development tools for video data curation and processing, including the NVIDIA Cosmos tokeniser and NVIDIA NeMo Curator for video processing.

The open-source Cosmos tokeniser provides robotics developers improved visual tokenisation by breaking down images and videos into high-quality tokens with exceptionally high compression rates. It runs up to 12x faster than current tokenisers, while NeMo Curator provides video processing curation up to 7x faster than unoptimised pipelines.

NVIDIA Isaac Lab is an open-source, robot learning framework built on NVIDIA Omniverse, a platform for developing OpenUSD applications for industrial digitalisation and physical AI simulation.

Developers can use Isaac Lab to train robot policies at scale. This open-source unified robot learning framework applies to any embodiment — from humanoids to quadrupeds to collaborative robots — to handle increasingly complex movements and interactions.

Project GR00T is an initiative to develop accelerated libraries, foundation models and data pipelines to accelerate the global humanoid robot developer ecosystem.

Six new Project GR00T workflows provide humanoid developers with blueprints to realise the most challenging humanoid robot capabilities. They include:

• GR00T-Gen for building generative AI-powered, OpenUSD-based 3D environments
• GR00T-Mimic for robot motion and trajectory generation
• GR00T-Dexterity for robot dexterous manipulation
• GR00T-Control for whole-body control
• GR00T-Mobility for robot locomotion and navigation
• GR00T-Perception for multimodal sensing

“Humanoid robots are the next wave of embodied AI,” said Jim Fan, senior research manager of embodied AI at NVIDIA. “NVIDIA research and engineering teams are collaborating across the company and our developer ecosystem to build Project GR00T to help advance the progress and development of global humanoid robot developers.”

NVIDIA Cosmos tokenisers provide encoding and decoding to simplify the development of world models. They set a new standard of minimal distortion and temporal instability, enabling high-quality video and image reconstructions.

1X, a humanoid robot company, has updated the 1X World Model Challenge dataset to use the Cosmos tokeniser. “NVIDIA Cosmos tokeniser achieves really high temporal and spatial compression of our data while still retaining visual fidelity,” said Eric Jang, vice president of AI at 1X Technologies. “This allows us to train world models with long horizon video generation in an even more compute-efficient manner.”

Other humanoid and general-purpose robot developers, including XPENG Robotics and Hillbot, are developing with the NVIDIA Cosmos tokeniser to manage high-resolution images and videos.

NeMo Curator now includes a video processing pipeline. This enables robot developers to improve their world-model accuracy by processing large-scale text, image and video data.