Kyutai’s AURA Talks Back in Real Time on a Single GPU

Human Verified

Share to: , and

A stylized human face formed from sound waves on a screen, speaking, with audio waveforms flowing through it

Kyutai has released AURA, a family of models that generate talking-head video from live audio, chunk by chunk, on a single graphics card. The smaller model runs in real time, while the larger one streams high-definition video at nearly ten frames per second. Both models are released with open weights.

How Streaming Works

Most talking-head models need several seconds of audio to start, because they look ahead and generate whole chunks. AURA generates video chunk by chunk with a rolling key-value cache, needing only the current chunk’s audio to begin. That removes the delay that makes other models unusable for live conversation.

The pipeline uses four denoising steps per chunk, far fewer than the dozens a diffusion model needs. A key-value cache is shared across all denoising timesteps, which keeps memory bounded however long the conversation runs. The result is a face that talks back as the words arrive, not minutes later.

Speed and Quality

AURA 1.3B streams 576p video in real time on one GPU at 20.7 frames per second, which makes it usable for interactive avatars. AURA 5B streams 720p at 9.7 frames per second, and the paper reports it runs 17 times faster than its TalkVerse teacher while matching it on quality.

The smaller model runs 23 times faster than its bidirectional teacher and reaches quality comparable to TalkVerse at a lower resolution. Both models were measured on talking-head benchmarks, Hallo3 and CelebV-HQ, using quality and audio-visual alignment scores. The paper says both are faster than every baseline in the single-GPU setting.

What Kyutai Released

Alongside the models, Kyutai released a dataset of 30,000 talking-head videos, 38.7 hours in total, with precomputed teacher trajectories for researchers who want to distil their own streaming models. The weights of both models are open, so anyone with a card can run them locally rather than through a service.

A live demo couples AURA 1.3B with Unmute, a voice assistant built on Kyutai’s speech models and a text model, letting users hold a spoken conversation with an avatar live on a single H100 card. The demo and video results are on Kyutai’s project page, as described in the paper.

Our Read

The talking face is the missing piece for voice assistants. Speech-to-speech models already hold conversations, but they only produce audio, while human conversation relies on faces, lip motion and expression. A model that streams a face in real time on one GPU turns a voice assistant into a presence.

The limits matter. All benchmarks are the authors’ evaluations, and the model’s quality matches its teacher only at a lower resolution. The demo and weights status is as stated in the paper, since the project page did not load when checked. Open weights plus a dataset make the claim testable.

Why It Matters

Real-time avatars matter for assistants, presenters and games, and until now capable models needed several cards or minutes per chunk. Krea Video puts video models in one browser studio, but AURA is the first open-weight model built for streaming, which is a different and harder problem than offline generation.

Background

Talking-head models such as TalkVerse and Wan-S2V produce results but rely on bidirectional attention, so they must generate chunks of several seconds at once. ByteDance’s Seedance family shows video generation advancing fast, yet AURA’s contribution is the streaming regime: causal, few-step, and small enough to run live on one card.

Source: arXiv paper and Kyutai project page

Share to: , and

Faheem R. Khattak

Senior automation engineer and AI marketing journo covering LLMs, algorithms, analytics, core updates, Bing search, and technical SEO.

0 Comments

Leave a Reply

Optional Take Tag:

🛡️ Spam protection: Honeypot + Time-Trap Active

This site uses Akismet to reduce spam. Learn how your comment data is processed.