DLSS 5: Generative Neural Rendering

[Paper]

We present DLSS 5, a real-time generative rendering stage for interactive graphics that complements conventional rendering with appearance priors learned from real-world visual data. Prior DLSS technologies use learned reconstruction to efficiently approximate outputs that would otherwise require a higher rendering budget. This improves rendering efficiency and image quality, but realism remains bounded by what the renderer can represent and simulate. In practice, real-time compute and memory budgets limit both authored scene representations and rendering algorithms, preventing them from fully capturing real-world appearance. Foundational generative models offer a complementary source of realism. They learn broad knowledge of real-world appearance and can synthesize complex visual effects that are difficult to represent faithfully with authored assets and real-time simulation, such as subsurface scattering in skin and light scattering through foliage.

However, directly applying these models to interactive rendering presents two fundamental challenges. First, general-purpose generative models do not reliably preserve developer-authored scene content and structure, even when their outputs appear visually plausible. Second, their inference cost exceeds the strict frame budgets of interactive rendering, particularly at 4K output resolution. DLSS 5 is designed to harness learned appearance priors while faithfully preserving developer-authored content and design intent. To achieve this within an interactive frame budget, we introduce 3D-guided neural rendering, a renderer-grounded generative approach. The system uses a one-step pixel-space diffusion model designed for high-resolution, real-time rendering. At inference, the model is conditioned on the current rendered frame, engine motion vectors, carried temporal state, and artistic-direction values. During training, consistency supervision from renderer-derived scene attributes further grounds generation to remain faithful to the authored scene. The resulting inference process is causal and deterministic, is trained for frame-to-frame temporal stability, and operates under a strict per-frame compute budget, enabling real-time rendering at up to 4K resolution.

DLSS 5 is the first DLSS technology to generate the final displayed appearance rather than reconstruct a higher-cost reference output from the conventional renderer. To the best of our knowledge it is also the first productized generative rendering model to operate in real time. DLSS 5 runs locally as a rendering stage within existing game pipelines on GeForce RTX 50 Series GPUs. Generative rendering does not replace physically based rendering or artistic authorship; it complements them by supplying learned real-world appearance knowledge that additional samples and explicitly authored scene representations alone cannot efficiently provide.