Overview
FlashDreams is a high-performance inference and serving library for interactive autoregressive video and world models. It began as the optimized runtime behind the OmniDreams closed-loop demo and has grown into a general platform for real-time world-model applications across driving simulation, robotics, games, virtual environments, and creative tools.
Interactive world models need more than one-shot video generation. A live application keeps receiving inputs, controls, and sensor updates while the model carries state forward and streams new outputs back. FlashDreams packages that loop into reusable pipelines, runners, model integrations, and serving components.
World-Model Serving
Offline video inference usually prepares a prompt or input image, runs the model, and returns a finished clip. Interactive world-model serving is a persistent session: initialize once, then repeatedly advance the world as new control input arrives.
That challenges the system design. The runtime has to manage latency, streaming cadence, GPU utilization, cache updates, and multi-GPU execution while the application remains responsive.
Offline video generation versus online autoregressive world-model serving.
Pipeline
FlashDreams exposes a streaming inference pipeline with three core phases: initialize persistent state, generate the next output chunk, and finalize cache updates for the next autoregressive step. Model integrations plug into this structure through encoders, diffusion models, schedulers, transformers, decoders, and caches.
FlashDreams streaming inference pipeline.
Examples
The repository includes first-party integrations for interactive and batch-style model families. Each model page includes setup instructions, runner names, command examples, and performance notes.
LingBot-World
Camera-controllable image-to-video world-model serving with a browser client.
OmniDreams
HDMap-conditioned driving world model for closed-loop simulation.
Self-Forcing
Streaming Wan2.1 text-to-video rollouts through the FlashDreams runner.
FlashVSR
Streaming video super-resolution in the same integration framework.
Supported Models
| Model | Use case |
|---|---|
| OmniDreams | HDMap-conditioned driving world-model generation. |
| LingBot-World | Camera-controllable image-to-video world-model serving. |
| Self-Forcing | Autoregressive Wan2.1 text-to-video generation. |
| Causal-Forcing | Streaming text-to-video and image-to-video generation. |
| Causal Wan2.2 | FastVideo Causal Wan2.2 14B MoE text-to-video generation. |
| FlashVSR | Streaming video super-resolution. |
| Wan2.1 | Bidirectional text-to-video and image-to-video generation. |
| Cosmos-Predict2.5 | Bidirectional text-to-video and image-to-video generation. |
Performance Highlights
FlashDreams focuses on the latency-critical parts of autoregressive world-model inference. The documentation includes matched-setting benchmark details and instructions for reproducing baseline comparisons.
See the model documentation for benchmark methodology, hardware settings, and reproduction notes.
Get Started
FlashDreams can be installed as a Python package or used from the source tree. The shortest source-based path is:
git clone https://github.com/NVIDIA/flashdreams.git
cd flashdreams
uv sync --extra dev --extra runners
export HF_TOKEN=<your-hf-token>
uv run flashdreams-run --helpFor a guided walkthrough, start with the quickstart. For new model integrations, see Add a new method.