Publications | Fundamental Generative AI Research

Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Models

Advances in Neural Information Processing Systems (NeurIPS)

Giannis Daras, Weili Nie, Karsten Kreis, Alex Dimakis, Morteza Mardani, Nikola Kovachki, Arash Vahdat

arXiv

Molecule Generation with Fragment Retrieval Augmentation

Advances in Neural Information Processing Systems (NeurIPS)

Seul Lee, Karsten Kreis, Srimukh Prasad Veccham, Meng Liu, Danny Reidenbach, Saee Paliwal, Arash Vahdat, Weili Nie

arXiv

L4GM: Large 4D Gaussian Reconstruction Model

Advances in Neural Information Processing Systems (NeurIPS)

Jiawei Ren, Kevin Xie, Ashkan Mirzaei, Hanxue Liang, Xiaohui Zeng, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, Huan Ling

arXiv

DiffUHaul: A Training-Free Method for Object Dragging in Images

ACM Transactions on Graphics (SIGGRAPH ASIA)

Omri Avrahami, Rinon Gal, Gal Chechik, Ohad Fried, Dani Lischinski, Arash Vahdat, Weili Nie

arXiv Website

BlobGEN-3D: Compositional 3D-Consistent Freeview Image Generation with 3D Blobs

ACM Transactions on Graphics (SIGGRAPH ASIA)

Chao Liu, Weili Nie, Sifei Liu, Abhishek Badki, Hang Su, Morteza Mardani, Benjamin Eckart, Arash Vahdat

pdf

Aligning Target-Aware Molecule Diffusion Models with Exact Energy Optimization

Advances in Neural Information Processing Systems (NeurIPS)

Siyi Gu, Minkai Xu, Alexander S Powers, Weili Nie, Tomas Geffner, Karsten Kreis, Jure Leskovec, Arash Vahdat, Stefano Ermon

arXiv

DiffiT: Diffusion Vision Transformers for Image Generation

European Conference on Computer Vision (ECCV)

Ali Hatamizadeh, Jiaming Song, Guilin Liu, Jan Kautz, Arash Vahdat

arXiv Website

DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents

International Conference on Machine Learning (ICML)

Yilun Xu, Gabriele Corso, Tommi Jaakkola, Arash Vahdat, Karsten Kreis

Website pdf code

Compositional Text-to-Image Generation with Dense Blob Representations

International Conference on Machine Learning (ICML)

Weili Nie, Sifei Liu, Morteza Mardani, Chao Liu, Benjamin Eckart, Arash Vahdat

arXiv Website

Align Your Steps: Optimizing Sampling Schedules in Diffusion Models

International Conference on Machine Learning (ICML)

Amirmojtaba Sabour, Sanja Fidler, Karsten Kreis

arXiv Website

Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automata

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Dongsu Zhang, Francis Williams, Zan Gojcic, Karsten Kreis, Sanja Fidler, Young Min Kim, Amlan Kar

arXiv

Dream-in-4D: A Unified Approach for Text- and Image-guided 4D Scene Generation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Yufeng Zheng, Xueting Li, Koki Nagano, Sifei Liu, Karsten Kreis, Otmar Hilliges, Shalini De Mello

arXiv Website

Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, Karsten Kreis

arXiv Website

WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space

International Conference on Learning Representations (ICLR)

Katja Schwarz, Seung Wook Kim, Jun Gao, Sanja Fidler, Andreas Geiger, Karsten Kreis

arXiv

Learning to Jointly Understand Visual and Tactile Signals

International Conference on Learning Representations (ICLR)

Yichen Li, Yilun Du, Chao Liu, Francis Williams, Michael Foshey, Benjamin Eckart, Jan Kautz, Joshua B. Tenenbaum, Antonio Torralba, Wojciech Matusik

pdf

A Variational Perspective on Solving Inverse Problems with Diffusion Models

International Conference on Learning Representations (ICLR)

Morteza Mardani, Jiaming Song, Jan Kautz, Arash Vahdat

arXiv

Fast Training of Diffusion Models with Masked Transformers

Transactions on Machine Learning Research

Hongkai Zheng, Weili Nie, Arash Vahdat, Anima Anandkumar

pdf

TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models

IEEE International Conference on Computer Vision (ICCV)

Tianshi Cao, Karsten Kreis, Sanja Fidler, Nicholas Sharp, Kangxue Yin

arXiv Website

SMRD: SURE-based Robust MRI Reconstruction with Diffusion Models

International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)

Batu Ozturkler, Chao Liu, Benjamin Eckart, Morteza Mardani, Jiaming Song, Jan Kautz

arXiv

PhysDiff: Physics-Guided Human Motion Diffusion Model

IEEE International Conference on Computer Vision (ICCV)
Oral

Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz

arXiv Website

DreamTeacher: Pretraining Image Backbones with Deep Generative Models

IEEE International Conference on Computer Vision (ICCV)

Daiqing Li, Huan Ling, Amlan Kar, David Acuna, Seung Wook Kim, Karsten Kreis, Antonio Torralba, Sanja Fidler

arXiv Website

Differentially Private Diffusion Models

Transactions on Machine Learning Research

Tim Dockhorn, Tianshi Cao, Arash Vahdat, Karsten Kreis

arXiv pdf

Real-Time Radiance Fields for Single-Image Portrait View Synthesis

ACM Transactions on Graphics (SIGGRAPH)

Alexander Trevithick, Matthew Chan, Michael Stengel, Eric Ryan Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano

arXiv Website

Loss-Guided Diffusion Models for Plug-and-Play Controllable Generation

International Conference on Machine Learning (ICML)

Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming-Yu Liu, Jan Kautz, Yongxin Chen, Arash Vahdat

pdf

I^2SB: Image-to-Image Schrödinger Bridge

International Conference on Machine Learning (ICML)

Guan-Horng Liu, Arash Vahdat, De-an Huang, Evangelos Theodorou, Weili Nie, Anima Anandkumar

arXiv Website

Fast Sampling of Diffusion Models via Operator Learning

International Conference on Machine Learning (ICML)

Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Azizzadenesheli, Anima Anandkumar

arXiv

Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Davis Rempe, Zhengyi Luo, Xue Bin Peng, Ye Yuan, Kris Kitani, Karsten Kreis, Sanja Fidler, Or Litany

arXiv Website

Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Paul Micaelli, Pavlo Molchanov, Arash Vahdat, Hongxu Yin, Jan Kautz

arXiv

Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Highlight

Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, Shalini De Mello

arXiv

NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Seung Wook Kim, Bradley Brown, Kangxue Yin, Karsten Kreis, Katja Schwarz, Daiqing Li, Robin Rombach, Antonio Torralba, Sanja Fidler

arXiv Website

Magic3D: High-Resolution Text-to-3D Content Creation

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, Tsung-Yi Lin

arXiv Website

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, Karsten Kreis

arXiv Website

Pseudoinverse-Guided Diffusion Models for Inverse Problems

International Conference on Learning Representations (ICLR)

Jiaming Song, Arash Vahdat, Morteza Mardani, Jan Kautz

Online Consistent Video Depth using Continuous Geometric Representations

International Conference on Robotics and Automation (ICRA)

Chao Liu, Benjamin Eckart, Jan Kautz

pdf

Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning

ArXiv Preprint

Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Korthikanti, Weili Nie, De-an Huang, Linxi Fan, Zhiding Yu, Shiyi Lan, Bo Li, Others

arXiv

Test-time prompt tuning for zero-shot generalization in vision-language models

Advances in Neural Information Processing Systems (NeurIPS)

Manli Shu, Weili Nie, De-an Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, Chaowei Xiao

arXiv website code

LION: Latent Point Diffusion Models for 3D Shape Generation

Advances in Neural Information Processing Systems (NeurIPS)

Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, Karsten Kreis

arXiv Website

GENIE: Higher-Order Denoising Diffusion Solvers

Advances in Neural Information Processing Systems (NeurIPS)

Tim Dockhorn, Arash Vahdat, Karsten Kreis

arXiv Website

LANA: Latency Aware Network Acceleration

European Conference on Computer Vision (ECCV)

Pavlo Molchanov, Jimmy Hall, Hongxu Yin, Jan Kautz, Nicolo Fusi, Arash Vahdat

arXiv

Diffusion Models for Adversarial Purification

International Conference on Machine Learning (ICML)

Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, Anima Anandkumar

arXiv Website

A-ViT: Adaptive Tokens for Efficient Vision Transformer

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Hongxu Yin, Arash Vahdat, Jose M. Alvarez, Arun Mallya, Jan Kautz, Pavlo Molchanov

arXiv Website

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

International Conference on Learning Representations (ICLR)

Zhisheng Xiao, Karsten Kreis, Arash Vahdat

arXiv Website

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion

International Conference on Learning Representations (ICLR)

Tim Dockhorn, Arash Vahdat, Karsten Kreis

arXiv Website

Neural Interferometry: Image Reconstruction from Astronomical Interferometers using Transformer-Conditioned Neural Fields

AAAI Conference on Artificial Intelligence (AAAI)

Ben Wu, Chao Liu, Benjamin Eckart, Jan Kautz

pdf

Score-based Generative Modeling in Latent Space

Advances in Neural Information Processing Systems (NeurIPS)

Arash Vahdat, Karsten Kreis, Jan Kautz

arXiv Website

Don't Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence

Advances in Neural Information Processing Systems (NeurIPS)

Tianshi Cao, Alex Bie, Arash Vahdat, Sanja Fidler, Karsten Kreis

arXiv Website

Controllable and Compositional Generation with Latent-Space Energy-Based Models

Advances in Neural Information Processing Systems (NeurIPS)

Weili Nie, Arash Vahdat, Anima Anandkumar

arXiv Website

A Contrastive Learning Approach for Training Variational Autoencoder Priors

Advances in Neural Information Processing Systems (NeurIPS)

Jyoti Aneja, Alexander Schwing, Jan Kautz, Arash Vahdat

arXiv

Self-Supervised Learning on 3D Point Clouds by Learning Discrete Generative Models

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Benjamin Eckart, Wentao Yuan, Chao Liu, Jan Kautz

pdf

See through Gradients: Image Batch Recovery via GradInversion

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M. Alvarez, Jan Kautz, Pavlo Molchanov

arXiv

VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models

International Conference on Learning Representations (ICLR)

Zhisheng Xiao, Karsten Kreis, Jan Kautz, Arash Vahdat

arXiv code

On the Distance between Two Neural Networks and the Stability of Learning

Advances in Neural Information Processing Systems (NeurIPS)

Jeremy Bernstein, Arash Vahdat, Yisong Yue, Ming-Yu Liu

arXiv code

NVAE: A Deep Hierarchical Variational Autoencoder

Advances in Neural Information Processing Systems (NeurIPS)

Arash Vahdat, Jan Kautz

arXiv Website

Neural FFTs for Universal Texture Image Synthesis

Advances in Neural Information Processing Systems (NeurIPS)

Morteza Mardani, Guilin Liu, Aysegul Dundar, Shiqiu Liu, Andrew Tao, Bryan Catanzaro

pdf

Contrastive Learning for Weakly Supervised Phrase Grounding

European Conference on Computer Vision (ECCV)
Spotlight

Tanmay Gupta, Arash Vahdat, Gal Chechik, Xiaodong Yang, Jan Kautz, Derek Hoiem

arXiv Website

Undirected Graphical Models as Approximate Posteriors

International Conference on Machine Learning (ICML)

Arash Vahdat, Evgeny Andriyash, William G Macready

arXiv

UNAS: Differentiable Architecture Search Meets Reinforcement Learning

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Arash Vahdat, Arun Mallya, Ming-Yu Liu, Jan Kautz

arXiv code

Semi-Supervised Semantic Image Segmentation with Self-correcting Networks

IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

Moustafa S. Ibrahim, Arash Vahdat, Mani Ranjbar, William G. Macready

arXiv