Runway has published a detailed account of how it serves Runway Characters, its product that turns a single image into a live character users can talk to, in real time. The research post, "Real-Time Video at Batch Size One," was released on November 5, 2026, alongside a paper titled "Runway Characters: Serving a 57B Video Diffusion Transformer for Real-Time Conversation."
The underlying model is a roughly 57-billion-parameter autoregressive video diffusion transformer that produces 704x1088 video at 24 fps, generating four-frame chunks every 166.7 milliseconds. Because sessions are one-on-one, Runway cannot batch requests across users, so every millisecond of per-step latency counts. At launch, the system took 240.9 ms per diffusion step on four H100s, too slow for real time.
After optimization, Runway reports 159.3 ms per step (1.51x faster), sustaining 24.5 fps per session on four H100s, and 147.4 ms with a fused communication kernel. The largest single gain came from cache upkeep, which fell from 69.7 ms to 6.2 ms per step; the original code spent 29% of GPU time there versus 6% on attention itself. Techniques include applying RoPE and FP8 quantization once as each frame enters the cache, a ring-buffer cache that advances without copying, a fused FP8 matmul and all-reduce kernel extended from ParallelKittens, and a dedicated GPU for the decoder so decoding overlaps with the next denoising step. On three B200s, the system runs at 154.3 ms per step in production.
Runway also shares fleet data: over seven weeks, workers averaged above 23.5 fps in 99.5% of sessions that streamed for at least a second. The post is candid about failures, describing an out-of-memory incident after a PyTorch and NCCL upgrade and noting that a single .item() call per step raised step time from 168.5 ms to 278 ms. Median end-to-end turn latency in Runway's traces was 2.13 seconds, of which the third-party voice agent accounted for 1.42 seconds and Runway's pipeline 0.49 seconds.
For teams building interactive generative video, the write-up is a rare production-level breakdown showing that serving cost at batch size one is dominated by memory movement and communication rather than attention compute, and that the voice layer, not the video model, is now the main latency bottleneck.