Differentiable rendering

Unbiased Differential Visibility Using Fixed-Step Walk-on-Spherical-Caps and Closest Silhouettes

Computing derivatives of path integrals under evolving scene geometry is a fundamental problem in physics-based differentiable rendering, which requires differentiating discontinuities in the visibility function. Warped-area reparameterization is a …

Radiance Surfaces: Optimizing Surface Representations with a 5D Radiance Field Loss

We present a fast and simple technique to convert images into a radiance surface-based scene representation. Building on existing radiance volume reconstruction algorithms, we introduce a subtle yet impactful modification of the loss function …

Amortizing Samples in Physics-Based Inverse Rendering using ReSTIR

Recently, great progress has been made in physics-based differentiable rendering. Existing differentiable rendering techniques typically focus on static scenes, but during inverse rendering---a key application for differentiable rendering---the scene …

SLANG.D: Fast, Modular and Differentiable Shader Programming

We introduce SLANG.D, a shading language that incorporates first-class automatic differentiation support derived from the Slang language. The new shading language allows us to transform a Direct3D-based path tracer to be fully differentiable with …

Flexible Isosurface Extraction for Gradient-Based Mesh Optimization

This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field, an increasingly common paradigm in applications including photogrammetry, generative …

Inverse Global Illumination Using a Neural Radiometric Prior

Inverse rendering methods that account for global illumination are becoming more popular, but current methods require evaluating and automatically differentiating millions of path integrals by tracing multiple light bounces, which remains expensive …

Recursive Control Variates for Inverse Rendering

We present a method for reducing errors---variance and bias---in physically based differentiable rendering (PBDR). Typical applications of PBDR repeatedly render a scene as part of an optimization loop involving gradient descent. The actual change …

Neural Fields meet Explicit Geometric Representations for Inverse Rendering of Urban Scenes

Reconstruction and intrinsic decomposition of scenes from captured imagery would enable many applications such as relighting and virtual object insertion. Recent NeRF based methods achieve impressive fidelity of 3D reconstruction, but bake the …

Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising

Recent advances in differentiable rendering have enabled high-quality reconstruction of 3D scenes from multi-view images. Most methods rely on simple rendering algorithms: pre-filtered direct lighting or learned representations of irradiance. We show …

Extracting Triangular 3D Models, Materials, and Lighting From Images

We present an efficient method for joint optimization of topology, materials and lighting from multi-view image observations. Unlike recent multi-view reconstruction approaches, which typically produce entangled 3D representations encoded in neural …