Efficient Dataflow Modeling of Peripheral Encoding in the Human Visual System

Computer graphics seeks to deliver compelling images, generated within a computing budget, targeted at a specific display device, and ultimately viewed by an individual user. The foveated nature of human vision offers an opportunity to efficiently allocate computation and compression to appropriate areas of the viewer’s visual field, of particular importance with the rise of high-resolution and wide field-of-view display devices.

Legate Sparse: Distributed Sparse Computing in Python

The sparse module of the popular SciPy Python library is widely used across applications in scientific computing, data analysis and machine learning. The standard implementation of SciPy is restricted to a single CPU and cannot take advantage of modern distributed and accelerated computing resources. We introduce Legate Sparse, a system that transparently distributes and accelerates unmodified sparse matrix-based SciPy programs across clusters of CPUs and GPUs, and composes with cuNumeric, a distributed NumPy library.

MesoGAN: Generative Neural Reflectance Shells

We introduce MesoGAN, a model for generative 3D neural textures. This new graphics primitive represents mesoscale appearance by combining the strengths of generative adversarial networks (StyleGAN) and volumetric neural field rendering. The primitive can be applied to surfaces as a neural reflectance shell; a thin volumetric layer above the surface with appearance parameters defined by a neural network.

VerilogEval: Evaluating Large Language Models for Verilog Code Generation

The increasing popularity of large language models (LLMs) has paved the way for their application in diverse domains. This paper proposes a benchmarking framework tailored specifically for evaluating LLM performance in the context of Verilog code generation for hardware design and verification. We present a comprehensive evaluation dataset consisting of 156 problems from the Verilog instructional website HDLBits. The evaluation set consists of a diverse set of Verilog code generation tasks, ranging from simple combinational circuits to complex finite state machines.

Ruth Rosenholtz

Ruth Rosenholtz joined NVIDIA Research in 2023, after a year as visiting scientist. Her research interests include behavioral experiments and computational modeling of human visual perception, and its applications. Particular vision topics include peripheral vision, visual attention, perceptual organization, material perception, and shape/depth perception. Applications include image quality, HCI, and vision for driving.

Rinon Gal

Rinon joined NVIDIA Research as an intern in June 2021 and has since been with the company. 

His research focuses on 2D generative models and their application to image and video editing. More recently, Rinon has been working on personalization of text-to-image models, where he aims to efficiently teach models how to generate images of unseen, user-provided concepts.

Boris Bonev

Boris Bonev received his PhD in applied mathematics from EPFL, numerical algorithms for large-scale PDE problems. At NVIDIA he works at the intersection of Machine Learning and classical numerical methods for Scientific Computing. He is excited about creating scalable algorithms from mathematical principles and mapping them to high-performance computing systems.

GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields

We propose GazeNeRF, a 3D-aware method for the task of gaze redirection. Existing gaze redirection methods operate on 2D images and struggle to generate 3D consistent results. Instead, we build on the intuition that the face region and eye balls are separate 3D structures that move in a coordinated yet independent fashion. Our method leverages recent advancements in conditional image-based neural radiance fields and proposes a two-branch architecture that predicts volumetric features for the face and eye regions separately.

Zero-shot Pose Transfer for Unrigged Stylized 3D Characters

Transferring the pose of a reference avatar to stylized 3D characters of various shapes is a fundamental task in computer graphics. Existing methods either require the stylized characters to be rigged, or they use the stylized character in the desired pose as ground truth at training. We present a zero-shot approach that requires only the widely available deformed non-stylized avatars in training, and deforms stylized characters of significantly different shapes at inference.