NVIDIA has opened up four new Cosmos 3 world models for anyone building self-driving cars and robots, publishing the weights for anyone to download and use commercially. The four checkpoints — Transfer-Auto, Sim-Auto, Sim-Transfer and Sim-Bimanual — are post-trained versions of the company’s Cosmos 3 Nano model, each with 16 billion trainable parameters. They generate the video and sensor data that driving and robotics teams need to train and test their systems without sending real vehicles or robots into every situation first.
## Four models, four jobs
Transfer-Auto is the workhorse of the set. It builds synthetic driving data, generating video across as many as 11 camera views, including fisheye lenses, with LiDAR readings. Road layouts and traffic come from World Scenario Map controls, while text prompts set the weather, lighting and time of day, so one road can replay as a rainy night or a morning.
The scale of one run shows what the model is for. NVIDIA’s run produces seven 720p videos totalling 297 frames, plus 99 LiDAR sweeps, from one driving scenario. The model supports clips of up to 400 frames at 480p or 720p, and teams can adapt it to their camera rigs and LiDAR setups by post-training it on their data.
Sim-Auto is the streaming version of the same idea. It is a distilled variant of Transfer-Auto that generates camera video and LiDAR in small chunks, a few denoising steps at a time, using what it has already generated plus updated driving controls.
That makes it the checkpoint for closed-loop testing, where a driving policy’s decisions feed back into the scene it sees next. Einride is already testing NVIDIA’s Alpamayo reasoning model in its self-driving trucks, showing how this kind of simulation feeds real autonomy work.
Sim-Transfer takes a physics simulator’s spatial controls — depth, segmentation, edges or a blurred layout — plus a text prompt, and renders photorealistic video while the simulator keeps hold of the physics. That division of labour matters: the simulator owns motion and contact, while the model only paints what the policy sees.
Sim-Bimanual predicts the next video frames directly from a two-armed robot’s actions, supporting interactive simulation for YAM and AgiBot robot bodies, driven by a policy or by human-operated joysticks. Start with one image, move the robot or the camera, and the model generates the next observation, so a single control sequence can switch between viewpoints and manipulation.
## One recipe, four stages
All four checkpoints share a four-stage training recipe that NVIDIA has also published. It starts with bidirectional adaptation on complete recorded clips, then teacher forcing, where each chunk is predicted from earlier observations. The recipe is the point as much as the weights: it shows teams how to adapt the models to their own sensors and domains.
Consistency distillation then compresses the model into a student that generates each chunk in four denoising steps, and a final self-forcing stage trains on the student’s own generated history so small errors do not compound. That last stage matters because a slightly mispredicted object becomes context for later frames, so errors can snowball without it.
The checkpoints sit inside the wider Cosmos Framework, which NVIDIA is extending beyond Cosmos 3 to cover Alpamayo, GR00T, Qwen and pi0.5 models. The framework adds reinforcement-learning pipelines and agent skills for training and inference. It also covers the full model lifecycle, from data processing and optimization to serving.
Cosmos NIM containers serve the models on GPUs, and NVIDIA says speculative decoding delivers roughly two to three times faster generation on H200 hardware. Generating with Nano needs 58 GiB of GPU memory in BF16, or 44 GiB in FP8.
Running the models is not light work. They need Linux, an NVIDIA GPU, Docker and the NVIDIA Container Toolkit, and NVIDIA’s published streaming setup was tested on a single GB200 rather than a minimum specification. That keeps the release aimed squarely at teams that already run NVIDIA hardware, rather than hobbyists on a single desktop card.
## Our read
What changes here is who gets to build world models. A year ago, interactive simulation at this level lived inside a handful of well-funded labs. Four open checkpoints with a commercial-use licence, plus the training recipes, hand the same starting point to any driving or robotics team with enough GPUs.
The timing is also a competitive statement. Google’s Genie 4 showed the previous week what a closed world model can do, and NVIDIA’s answer is to open its weights and sell the platform around them. That is a familiar NVIDIA play: give away the models, and the GPUs, frameworks and NIM containers that run them become the product.
Open weights also let researchers compare methods on a shared starting point, which is how a field moves faster. The bottleneck shifts from access to the models to the quality of a team’s own data and sensor rigs, and to proving that skills learned in simulation survive the real world.
The claims still need a sceptical eye. Every performance figure comes from NVIDIA itself, and the company is candid that the work ahead is to measure where the models faithfully support policy learning and evaluation. Generated video that looks right is not proof that a robot trained on it will behave right in the real world.
The agent-compute side of the same bet is visible in the Vera CPU, built to run the agents that will consume these simulated worlds. Simulation is only half the stack; the other half is the hardware that turns simulated practice into real decisions.
## Why It Matters
Driving and robotics teams spend enormous effort collecting edge cases: the rainy night, the obscured sign, the unusual pedestrian. Synthetic data generated on demand makes those cases cheap to produce and safe to test, and synchronized camera-plus-LiDAR output means perception and planning can train on the same scene.
If the models hold up in practice, they cut months off the loop between an idea and a tested driving policy. The release also matters for smaller teams: a lab that cannot afford a fleet of test vehicles or a warehouse of robots can now generate training scenes on rented GPUs instead.
That lowers the entry cost for physical AI, but it also raises the stakes on validation, since a team that trusts synthetic data too far will only discover the gap when its system meets the real world. Cheap data is only useful if it is honest data.
## Background
Cosmos 3 has been building toward this release all year. NVIDIA released the compact Cosmos 3 Edge model for on-device robots in July, followed by the Cosmos3-H-Dreams surgical world model in October, and a set of Cosmos data factory blueprints and training tools the same month.
Cosmos models passed 10 million downloads on Hugging Face in July, making the platform one of the most widely used open starting points for physical AI. The four new checkpoints extend that platform from understanding the world to simulating it interactively.
Source: [NVIDIA’s Cosmos 3 announcement](https://huggingface.co/blog/nvidia/cosmos-interactive-world-simulation) and [NVIDIA’s technical blog](https://developer.nvidia.com/blog/build-world-models-for-interactive-simulation-with-nvidia-cosmos/)
0 Comments