Publications

Our publications provide insight into some of our leading-edge research.

Filters

Filters

Publication Year

Research Areas

Events

2025

Play4D: Accelerated and Interactive Free-viewpoint Video Streaming for Virtual Reality and Light Field Displays
Jonghyun Kim, Michael Stengel, Amrita Mazumdar, Tianye Li, Cheng Sun, David Luebke, Shalini De Mello
RaySt3R: Predicting Novel Depth Maps for Zero-Shot Object Completion
Bardienus P. Duisterhof, Jan Oberst, Bowen Wen, Stan Birchfield, Deva Ramanan, Jeffrey Ichnowski
Attention on the Sphere
Boris Bonev, Max Rietmann, Andrea Paris, Alberto Carpentieri, Thorsten Kurth
Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation
Riccardo Corvi, Davide Cozzolino, Ekta Prashnani, Shalini De Mello, Koki Nagano, Luisa Verdoliva
Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
Marco Pavone, Many other contributors found on Page 33
Task-Oriented Human Grasp Synthesis via Context- and Task-Aware Diffusers
An-Lun Liu, Yu-Wei Chao, Yi-Ting Chen
Pedestrian Collision Avoidance in Hemianopia during Natural Walking in Immersive Virtual Reality
Jonathan K. Doyon, Sujin Kim, Alex D. Hwang, Jae-Hyun Jung
Real-time 3D Visualization of Radiance Fields on Light Field Displays
Jonghyun Kim, Cheng Sun, Michael Stengel, Matthew Chan, Andrew Russell, Jaehyun Jung, Wil Braithewaite, Shalini De Mello, David Luebke
GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control
Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alex Keller, Sanja Fidler, Jun Gao
Radiance Surfaces: Optimizing Surface Representations with a 5D Radiance Field Loss
Ziyi Zhang, Nicolas Roussel, Thomas Müller, Tizian Zeltner, Merlin Nimier-David, Fabrice Rousselle, Wenzel Jakob
Identity-Motion Trade-offs in Text-to-Video Generation
Yuval Atzmon, Rinon Gal, Yoad Tewel, Yoni Kasten, Gal Chechik
FoundationStereo: Zero-Shot Stereo Matching
Bowen Wen, Matthew Trepte, Joseph Aribido, Jan Kautz, Orazio Gallo, Stan Birchfield
Best Paper Nomination
Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholds
Eitan Shaar, Ariel Shaulov, Gal Chechik, Lior Wolf
RL-RC-DoT: A Block-level RL agent for Task-Aware Video Compression
Uri Gadot, Assaf Shocher, Shie Mannor, Gal Chechik, Assaf Hallak
TriTex: Learning Texture from a Single Mesh via Triplane Semantic Features
Dana Cohen-Bar, Daniel Cohen-Or, Gal Chechik, Yoni Kasten
BLADE: Single-view Body Mesh Estimation through Accurate Depth Estimation
Shengze Wang, Jiefeng Li, Tianye Li, Ye Yuan, Henry Fuchs, Koki Nagano, Shalini De Mello, Michael Stengel
Coherent 3D Portrait Video Reconstruction via Triplane Fusion
Shengze Wang, Xueting Li, Chao Liu, Matthew Chan, Michael Stengel, Henry Fuchs, Shalini De Mello, Koki Nagano
SimAvatar: Simulation-Ready Clothed Gaussian Avatars from Text
Xueting Li, Ye Yuan, Shalini De Mello, Gilles Daviet, Jonathan Leaf, Miles Macklin, Jan Kautz, Umar Iqbal
GRS: Generating robotic simulation tasks from real-world images
Alex Zook, Josef Spjut, Jonathan Tremblay
MambaVision: A Hybrid Mamba-Transformer Vision Backbone
Ali Hatamizadeh, Jan Kautz
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, Stan Birchfield
SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulation
Cheng-Chun Hsu, Bowen Wen, Jie Xu, Yashraj Narang, , Yuke Zhu, Joydeep Biswas, Stan Birchfield
AI 3D Selfie: Real-Time Single-Image 3D Face Reconstruction for Light-Field Displays
Jonghyun Kim, Michael Stengel, Matthew Chan, Koki Nagano, Shalini De Mello, David Luebke
LongVILA: Scaling Long-Context Visual Language Models for Long Videos
Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Haotian Tang, Shang Yang, Zhijian Liu, Ethan He, Hongxu Yin, Pavlo Molchanov, Jan Kautz, Linxi Fan, Yuke Zhu, Yao Lu (Jason), Song Han
LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models
Zhengyi Wang, Jonathan Lorraine, Yikai Wang, Hang Su, Jun Zhu, Sanja Fidler, Xiaohui Zeng
Multi-student Diffusion Distillation for Better One-step Generators
Yanke Song, Jonathan Lorraine, Weili Nie, Karsten Kreis, James Lucas
Spatio-Temporal Context Prompting for Zero-Shot Action Detection
Wei-Jhe Huang, Min-Hung Chen, Shang-Hong Lai
Semantic Prompt Learning for Weakly-Supervised Semantic Segmentation
Ci-Siang Lin, Chien-Yi Wang, Frank Wang, Min-Hung Chen
CorrFill: Enhancing Faithfulness in Reference-based Inpainting with Correspondence Guidance in Diffusion Models
Kuan-Hung Liu, Cheng-Kun Yang, Min-Hung Chen, Yu-Lun Liu, Yen-Yu Lin

2024

L4GM: Large 4D Gaussian Reconstruction Model
Jiawei Ren, Kevin Xie, Ashkan Mirzaei, Hanxue Liang, Xiaohui Zeng, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, Huan Ling
Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Models
Giannis Daras, Weili Nie, Karsten Kreis, Alexandros G. Dimakis, Morteza Mardani, Nikola Kovachki, Arash Vahdat
QUEEN: QUantized Efficient ENcoding for Streaming Free-viewpoint Videos
Sharath Girish, Tianye Li, Amrita Mazumdar, Abhinav Shrivastava, David Luebke, Shalini De Mello