MambaVision: A Hybrid Mamba-Transformer Vision Backbone

We propose a novel hybrid Mamba-Transformer backbone, MambaVision, specifically tailored for vision applications. Our core contribution includes redesigning the Mamba formulation to enhance its capability for efficient modeling of visual features. Through a comprehensive ablation study, we demonstrate the feasibility of integrating Vision Transformers (ViT) with Mamba. Our results show that equipping the Mamba architecture with self-attention blocks in the final layers greatly improves its capacity to capture long-range spatial dependencies.

Marco: Configurable Graph-Based Task Solving and Multi-AI Agents Framework for Hardware Design

Hardware design presents numerous challenges stemming from its complexity and advancing technologies. These challenges result in longer turn-around-time (TAT) for optimizing performance, power, area, and cost (PPAC) during synthesis, verification, physical design, and reliability loops. Large Language Models (LLMs) have shown remarkable capacity to comprehend and generate natural language at a massive scale, leading to many potential applications and benefits across various domains.

Alán Aspuru-Guzik

Senior Director of Quantum Chemistry.

 aspuru@nvidia.com  Toronto, Canada

My research at NVIDIA focuses at the intersection of Quantum Computing, Artificial Intelligence and Chemical applications. 

-- Near-term quantum algorithm development

-- Algorithms for fast quantum chemistry simulation for quantum computers and classical computers

-- Self-driving laboratories and robotics for chemical automation

-- Chemical generative models for materials design and drug discovery

Experimenting with Artificial Intelligence: Programming Pathfinding Algorithms in C++ with Unreal Engine 5

Recent years have seen a rise in machine learning applications deployed in different technological contexts. It has shaped how we experience technology and led to new doors of innovation and a mass market interest in artificial intelligence (AI). While academic research pushes the boundaries of AI through novel approaches, games serve as important testbeds for virtual agents. Game AI focuses on human psychology by using and looking for solutions to create fun illusions of intelligence.

Studying Esports Competition: Piloting Methodology for User Studies During Tournaments

Designing experiments is a well-studied, complex task with many conflicting constraints. In experimental user studies, the overarching goal is to gain understanding of how a system affects users by manipulating independent variables (eg, an interface's configuration) while measuring dependent variables (eg, the user's performance).

Student-T and Beyond: Practical Tools for Multiple-Scattering BSDFs with General NDFs

We present a practical importance-sampling scheme for the Student-T distribution of visible normals by representing the Student-T NDF as a superposition of Beckmann NDFs. Additionally, we derive a new form of delta tracking to evaluate and sample exact BSDFs with general full-sphere NDFs. These tools permit efficient computation of benchmark BSDF values for the multiple scattering from general (including porous) rough surfaces.

An Approximate Mie Scattering Function for Fog and Cloud Rendering

The Mie phase function describes the complex shapes that arise when light is scattered by water droplets. Inconvenient tables of data are required to include Mie scattering in a path tracer. To avoid this complexity, analytic models such as Cornette-Shanks (CS) or Henyey-Greenstein (HG) mixtures are often used instead, resulting in a lack of accuracy for fog, clouds, skies and tissue. We show that a blend of HG and Draine's phase function can accurately match 95% of the Mie phase function over a wide range of droplet sizes.

On Smoothly Varying Frame Timing in First-Person Gaming

In video games, the time between rendered frames varies significantly and is often uncontrolled. With the advent of variable refresh rate (VRR) displays, like G-SYNC and FreeSync, players can experience varying frame timing as produced instead of aliasing to nearby refresh timings (e.g. VSYNC On). We investigate how the presentation of smoothly varying frame times over time affects a gamer's perception of smoothness and aiming performance. Participants performed a first person targeting task while experiencing sinusoidally varying frame times of different amplitude and mean duration.

Gated Delta Networks: Improving Mamba2 with Delta Rule

Linear Transformers have gained attention as efficient alternatives to standard Transformers, but their performance in retrieval and long-context tasks has been limited. To address these limitations, recent work has explored two distinct mechanisms: gating for adaptive memory control and the delta update rule for precise memory modifications. We observe that these mechanisms are complementary: gating enables rapid memory erasure while the delta rule facilitates targeted updates.

SpecTrack: Learned Multi-Rotation Tracking via Speckle Imaging

Precision pose detection is increasingly demanded in fields such as personal fabrication, Virtual Reality (VR), and robotics due to its critical role in ensuring accurate positioning information. However, conventional vision-based systems used in these systems often struggle with achieving high precision and accuracy, particularly when dealing with complex environments or fast-moving objects. To address these limitations, we investigate Laser Speckle Imaging (LSI), an emerging optical tracking method that offers promising potential for improving pose estimation accuracy.