Publications

Our publications provide insight into some of our leading-edge research.

Filters

Filters

Publication Year

Research Areas

Events

2025

Fugatto 1 - Foundational Generative Audio Transformer Opus 1
Rafael Valle, Rohan Badlani, Zhifeng Kong, Sang-gil Lee, Arushi Goel, Sungwon Kim, Joao Felipe Santos, Shuqi Dai, Siddharth Gururani, Aya AIJa'fari, Alex Liu, Kevin Shih, Wei Ping, Huck Yang, Bryan Catanzaro
Gated Delta Networks: Improving Mamba2 with Delta Rule
Songlin Yang, Jan Kautz, Ali Hatamizadeh
Audio Large Language Models Can Be Descriptive Speech Quality Evaluators
Chen Chen, Yuchen Hu, Siyin Wang, Helin Wang, Zhehuai Chen, Chao Zhang, Huck Yang, EngSiong Chng
UniWav: Towards Unified Pre-training for Speech Representation Learning and Generation
Alexander H. Liu, Sang-gil Lee, Huck Yang, Yuan Gong, Frank Wang, James R. Glas, Rafael Valle
Towards Neural Scaling Laws for Time Series Foundation Models
Qingren Yao, Huck Yang, Renhe Jiang, Ming Jin, Shirui Pan
Energy-Based Diffusion Language Models for Text Generation
Minkai Xu, Tomas Geffner, Karsten Kreis, Weili Nie, Yilun Xu, Jure Leskovec, Stefano Ermon, Arash Vahdat
Truncated Consistency Models
Sangyun Lee, Yilun Xu, Tomas Geffner, Giulia Fanti, Karsten Kreis, Arash Vahdat, Weili Nie
Proteina: Scaling Flow-based Protein Structure Generative Models
Tomas Geffner, Kieran Didi, Zuobai Zhang, Danny Reidenbach, Zhonglin Cao, Jason Yim, Mario Geiger, Christian Dallago, Emine Kucukbenli, Arash Vahdat, Karsten Kreis
ProtComposer: Compositional Protein Structure Generation with 3D Ellipsoids
Hannes Stark, Bowen Jing, Tomas Geffner, Jason Yim, Tommi Jaakkola, Arash Vahdat, Karsten Kreis

2024

FasterViT: Fast Vision Transformers with Hierarchical Attention
Ali Hatamizadeh, Greg Heinrich, Hongxu Danny Yin, Andrew Tao, Jose M. Alvarez, Jan Kautz, Pavlo Molchanov
WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space
Katja Schwarz, Seung Wook Kim, Jun Gao, Sanja Fidler, Andreas Geiger, Karsten Kreis
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
YuChen Hu, Chen Chen, Huck Yang, Ruizhe Li, Chao Zhang, Pin-Yu Chen, EnSiong Chng
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition
Chen Chen, Ruizhe Li, Yuchen Hu, Sabato Marco Siniscalchi, Pin-Yu Chen, Ensiong Chng, Huck Yang

2023

Graph Metanetworks for Processing Diverse Neural Architectures
Derek Lim, Haggai Maron, Marc T. Law, Jonathan Lorraine, James Lucas
GPViT: A High Resolution Non-Hierarchical Vision Transformer with Group Propagation
Chenhongyi Yang, Jiarui Xu, Shalini De Mello, Elliot J. Crowley, Xiaolong Wang
Notable top 25%
Oral
Robust and Controllable Object-Centric Learning through Energy-based Models
Ruixiang Zhang, Gerry Che, Boris Ivanovic, Renhao Wang, Marco Pavone, Yoshua Bengio, Liam Paull

2022

Robust and Controllable Object-Centric Learning through Energy-based Models
Ruixiang Zhang, Tong Che, Boris Ivanovic, Renhao Wang, Marco Pavone, Yoshua Bengio, Liam Paull
On Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning
Guy Tenneholtz, Assaf Hallak, Gal Dalal, Shie Mannor, Gal Chechik
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, Daniel Cohen-Or
Top 25%
Accelerated Policy Learning with Parallel Differentiable Simulation
Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojciech Matusik, Animesh Garg, Miles Macklin
Score-Based Generative Modeling with Critically-Damped Langevin Diffusion
Tim Dockhorn, Arash Vahdat, Karsten Kreis
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
Zhisheng Xiao, Karsten Kreis, Arash Vahdat

2021

VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
Zhisheng Xiao, Karsten Kreis, Jan Kautz, Arash Vahdat