Publications

Our publications provide insight into some of our leading-edge research.

Filters

Filters

Publication Year

Research Areas

Events

2024

SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
Zihan Zhou, Animesh Garg, Dieter Fox, Caelan Garrett, Ajay Mandlekar
SkillGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment
Caelan Garrett, Ajay Mandlekar, Bowen Wen, Dieter Fox
NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptors
Shuo Cheng, Caelan Garrett, Ajay Mandlekar, Danfei Xu
HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation
Yi Li, Yuquan Deng, Jesse Zhang, Joel Jang, Marius Memmel, Caelan Garrett, Fabio Ramos, Dieter Fox, Anqi Li, Abhishek Gupta, Ankit Goyal
Guiding Long-Horizon Task and Motion Planning with Vision Language Models
Zhutian Yang, Caelan Garrett, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints
Nishanth Kumar, William Shen, Fabio Ramos, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Caelan Garrett
Constructability-driven design of frame structures with state-space search methods
Yijiang Huang, Caelan Garrett, Caitlin Mueller
ReMatching Dynamic Reconstruction Flow
Sara Oblak, Despoina Paschalidou, Sanja Fidler, Matan Atzmon
Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learning
Ryan Hoque, Ajay Mandlekar, Caelan Garrett, Ken Goldberg, Dieter Fox
DiMSam: Diffusion Models as Samplers for Task and Motion Planning under Partial Observability
Xiaolin Fang, Caelan Garrett, Clemens Eppner, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Dieter Fox
Best Conference Paper Finalist
Best Student Paper Finalist
DiffiT: Diffusion Vision Transformers for Image Generation
Ali Hatamizadeh, Jiaming Song, Guilin Liu, Jan Kautz, Arash Vahdat
Learning to Move Like Professional Counter-Strike Players
David Durst, F. Xie, V. Sarukkai, Brennan Shacklett, Iuri Frosio, Chen Tessler, Joohwan Kim, C. Taylor, G. Bernstein, S. Choudhury, P. Hanrahan,, Kayvon Fatahalian
Conformer without Convolutions
Matthijs Van keirsbilck, Alex Keller
LLM Pruning and Distillation in Practice: The Minitron Approach
Sharath Turuvekere Sreenivas, Saurav Muralidharan, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, Pavlo Molchanov
Kilometer-Scale Convection Allowing Model Emulation using Generative Diffusion Modeling
Jaideep Pathak, Yair Cohen, Piyush Garg, Peter Harrington, Noah Brenowitz, Dale Durran, Morteza Mardani, Arash Vahdat, Shaoming Xu, Karthik Kashinath, Mike Pritchard
VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool
Chia-Tung (Mark) Ho, Haoxing (Mark) Ren, Brucek Khailany
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
Yuchen Hu, Chen Chen, Huck Yang, Ruizhe Li, Zhehuai Chen, Eng Siong Chng
Align Your Steps: Optimizing Sampling Schedules in Diffusion Models
Amirmojtaba Sabour, Sanja Fidler, Karsten Kreis
DoRA: Weight-Decomposed Low-Rank Adaptation
Shih-Yang Liu, Chien-Yi Wang, Hongxu Danny Yin, Pavlo Molchanov, Frank Wang, Kwang-Ting Cheng, Min-Hung Chen
DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents
Yilun Xu, Gabriele Corso, Tommi Jaakkola, Arash Vahdat, Karsten Kreis
Experimenting with Artificial Intelligence: Programming Pathfinding Algorithms in C++ with Unreal Engine 5
Deborah Yuen, Josef Spjut
Improving Hyperparameter Optimization with Checkpointed Model Weights
Nikhil Mehta, Jonathan Lorraine, Steve Masson, Ramanathan Arunachalam, Zaid Pervaiz Bhat, James Lucas, Arun George Zachariah
Large Language Model (LLM) for Standard Cell Layout Design Optimization
Chia-Tung (Mark) Ho, Haoxing (Mark) Ren
Best Paper Award
Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models
Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, Karsten Kreis
Outdoor Scene Extrapolation with Hierarchical Generative Cellular Automata
Dongsu Zhang, Francis Williams, Zan Gojcic, Karsten Kreis, Sanja Fidler, Young Min Kim, Amlan Kar
Nemotron-4 340B
An Empirical Study of Mamba-based Language Models
Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, Garvit Kulshreshtha, Vartika Singh, Jared Casper, Jan Kautz, Mohammad Shoeybi, Bryan Catanzaro
Full-colour 3D holographic augmented-reality displays with metasurface waveguides
Manu Gopakumar, Gun-Yeal Lee, Suyeon Choi, Brian Chao, Yifan Peng, Jonghyun Kim, Gordon Wetzstein
FasterViT: Fast Vision Transformers with Hierarchical Attention
Ali Hatamizadeh, Greg Heinrich, Hongxu Danny Yin, Andrew Tao, Jose M. Alvarez, Jan Kautz, Pavlo Molchanov
WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space
Katja Schwarz, Seung Wook Kim, Jun Gao, Sanja Fidler, Andreas Geiger, Karsten Kreis
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
YuChen Hu, Chen Chen, Huck Yang, Ruizhe Li, Chao Zhang, Pin-Yu Chen, EnSiong Chng
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition
Chen Chen, Ruizhe Li, Yuchen Hu, Sabato Marco Siniscalchi, Pin-Yu Chen, Ensiong Chng, Huck Yang