Blogs

A technical survey of autoregressive video generation: why causality enables streaming and KV-cache reuse, how few-step distillation makes it practical, and how recent systems generate longer videos while reducing training cost.

Four-bit floating point is moving from a storage-only compression trick to a primitive for training and inference across LLMs, diffusion, video generation, KV cache, and attention.

Why KV cache compression methods that work on paper fail in production — FlashAttention hides attention scores, and repeated token eviction frees no paged GPU memory — and how a geometric property hidden beneath RoPE resolves both.

Long-video training changes the unit of distributed computation. This blog explains how sequence parallelism scales training when one video sample is too long for one GPU, comparing LongVILA MM-SP and LongLive-2.0 Balanced SP.

Video generation is becoming an infrastructure problem: long context, memory, decoding, scheduling, compression, parallelism, and deployment-aware design.