Research Areas

Computer Vision

Associated Publications

2023

Point-Cloud Completion with Pretrained Text-to-image Diffusion Models
Yoni Kasten, Ohad Rahamim, Gal Chechik
NeurIPS 2023
SceneScape: Text-Driven Consistent Scene Generation
Rafail Fridman, Amit Abecasis, Yoni Kasten, Tali Dekel
NeurIPS 2023
Neural LiDAR Fields for Novel View Synthesis
Shengyu Huang, Zan Gojcic, Zian Wang, Francis Williams, Yoni Kasten, Sanja Fidler, Konrad Schindler, Or Litany
ICCV 2023
Syntactic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment
Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, Gal Chechik
NeurIPS 2023
Oral presentation
HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions
Andrew Guo, Bowen Wen, Jianhe Yuan, Jonathan Tremblay, Stephen Tyree, Jeff Smith, Stan Birchfield
IROS 2023
Norm-guided latent space exploration for text-to-image generation
Dvir Samuel, Rami Ben-Ari, Nir Darshan, Haggai Maron, Gal Chechik
NeurIPS 2023
Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection
Yazhou Xing, Amrita Mazumdar, Anjul Patney, Chao Liu, Hongxu Danny Yin, Qifeng Chen, Jan Kautz, Iuri Frosio
Arxiv
Flexible Isosurface Extraction for Gradient-Based Mesh Optimization
Tianchang Shen, Jacob Munkberg, Jon Hasselgren, Kangxue Yin, Zian Wang, Wenzheng Chen, Zan Gojcic, Sanja Fidler, Nicholas Sharp, Jun Gao
ACM Transactions On Graphics (SIGGRAPH 2023)
Live 3D Portrait: Real-Time Radiance Fields for Single-Image Portrait View Synthesis
Alexander Trevithick, Matthew Chan, Michael Stengel, Eric R. Chan, Chao Liu, Zhiding Yu, Sameh Khamis, Manmohan Chandraker, Ravi Ramamoorthi, Koki Nagano
ACM Transactions On Graphics (SIGGRAPH 2023)
Learning Physically Simulated Tennis Players from Broadcast Videos
Haotian Zhang, Ye Yuan, Viktor Makoviychuk, Yunrong Guo, Sanja Fidler, Jason Peng, Kayvon Fatahalian
SIGGRAPH 2023 (Best Paper Honorable Mention)
SSIF: Single-shot Implicit Morphable Faces With Consistent Texture Parameterization
Connor Zhizhen Lin, Koki Nagano, Jan Kautz, Eric R. Chan, Umar Iqbal, Leonidas Guibas, Gordon Wetzstein, Sameh Khamis
SIGGRAPH 2023
Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles
Pasquale Antonante, Sushant Veer, Karen Leung, Xinshuo Weng, Luca Carlone, Marco Pavone
Robotics: Science and Systems (RSS) 2023
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
Yuzhe Qin, Wei Yang, Binghao Huang, Karl Van Wyk, Hao Su, Xiaolong Wang, Yu-Wei Chao, Dieter Fox
Robotics: Science and Systems (RSS) 2023
Affordance Diffusion: Synthesizing Hand-Object Interactions
Yufei Ye, Xueting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song, Shubham Tulsiani, Sifei Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation
Taeyeop Lee, Jonathan Tremblay, Valts Blukis, Bowen Wen, Byeong-Uk Lee, Inkyu Shin, Stan Birchfield, In So Kweon, Kuk-Jin Yoon
CVPR 2023
Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagation
Heng Yang, Marco Pavone
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Selected as a Highlight Paper
FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization
Jiawei Yang, Marco Pavone, Yue Wang
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Neural Congealing: Aligning Images to a Joint Semantic Atlas
Dolev Ofri-Amar, Michal Geyer, Yoni Kasten, Tali Dekel
CVPR 2023
Learning Human-to-Robot Handovers from Point Clouds
Sammy Christen, Wei Yang, Claudia Pérez D’Arpino , Otmar Hilliges, Dieter Fox, Yu-Wei Chao
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Highlight
Zero-shot Pose Transfer for Unrigged Stylized 3D Characters
Jiashun Wang, Xueting Li, Sifei Liu, Shalini De Mello, Orazio Gallo, Xiaolong Wang, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields
Alessandro Ruzzi, Xiangwei Shi, Xi Wang, Gengyan Li, Shalini De Mello, Hyung Jin Chang, Xucong Zhang, Otmar Hilliges
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models
Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, Shalini De Mello
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Hightlight top 10%
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas Müller, Alex Evans, Dieter Fox, Jan Kautz, Stan Birchfield
CVPR 2023
Neuralangelo: High-Fidelity Neural Surface Reconstruction
Max Zhaoshuo Li, Thomas Müller, Alex Evans, Russell H. Taylor, Mathias Unberath, Ming-Yu Liu, Chen-Hsuan Lin
CVPR 2023
The Best Inventions of 2023, TIME Magazine
Magic3D: High-Resolution Text-to-3D Content Creation
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, Tsung-Yi Lin
CVPR 2023 (highlight)
Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifiers
Yixuan Huang, Adam Conkey, Tucker Hermans
IEEE International Conference on Robotics and Automation (ICRA)
Planning with Occluded Traffic Agents using Bi-Level Variational Occlusion Models
Filippos Christianos, Peter Karkus, Boris Ivanovic, Stefano V. Albrecht, Marco Pavone
IEEE International Conference on Robotics and Automation (ICRA) 2023
FewSOL: A Dataset for Few-Shot Object Learning in Robotic Environments
Jishnu Jaykumar P, Yu-Wei Chao, Yu Xiang
IEEE International Conference on Robotics and Automation (ICRA) 2023
Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation
Yunzhi Lin, Thomas Müller, Jonathan Tremblay, Bowen Wen, Stephen Tyree, Alex Evans, Patricio A. Vela, Stan Birchfield
IEEE International Conference on Robotics and Automation (ICRA) 2023
The Best Defense is a Good Offense: Adversarial Augmentation against Adversarial Attacks
Iuri Frosio, Jan Kautz
CVPR 2023
RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control
Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay, Bowen Wen, Ye Yuan, Stephen Tyree, Charles Loop, Alexander Schwing, Stan Birchfield
ICRA 2023
Subpixel Deblurring of Anti-Aliased Raster Clip Art
Jinfan Yang, Nicholas Vining, Shakiba Kheradmand, Nathan Carr, Leonid Sigal, Alla Sheffer
Computer Graphics Forum (Proc. Eurographics 2023)
GPViT: A High Resolution Non-Hierarchical Vision Transformer with Group Propagation
Chenhongyi Yang, Jiarui Xu, Shalini De Mello, Elliot J. Crowley, Xiaolong Wang
International Conference on Learning Representations (ICLR) 2023
Notable top 25%, Oral
Robust and Controllable Object-Centric Learning through Energy-based Models
Ruixiang Zhang, Gerry Che, Boris Ivanovic, Renhao Wang, Marco Pavone, Yoshua Bengio, Liam Paull
International Conference on Learning Representations (ICLR) 2023
Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis
Wan-Cyuan Fan, Yen-Chun Chen, Dongdong Chen, Yu Cheng, Lu Yuan, Frank Wang
AAAI 2023
Target-free Text-guided Image Manipulation
Wan-Cyuan Fan, Cheng-Fu Yang, Chiao-An Yang, Frank Wang
AAAI 2023
Self-Supervised Pyramid Representation Learning for Multi-Label Visual Analysis and Beyond
Cheng-Yen Hsieh, Chih-Jung Chang, Fu-En Yang, Frank Wang
WACV 2023

2022

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles
Alec Farid, Sushant Veer, Boris Ivanovic, Karen Leung, Marco Pavone
Conference on Robot Learning (CoRL) 2022
Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentation
Qiuyu Chen, Karl Van Wyk, Yu-Wei Chao, Wei Yang, Arsalan Mousavian, Abhishek Gupta, Dieter Fox
The Conference on Robot Learning (CoRL) 2022
Robust Trajectory Prediction against Adversarial Attacks
Yulong Cao, Danfei Xu, Xinshuo Weng, Z. Morely Mao, Anima Anandkumar, Chaowei Xiao, Marco Pavone
Conference on Robot Learning (CoRL) 2022
Selected for Oral Presentation
MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare
Yann Labbe, Lucas Manuelli, Arsalan Mousavian, Stephen Tyree, Stan Birchfield, Jonathan Tremblay, et al.
CoRL 2022
Motion Policy Networks
Adam Fishman, Adithya Murali, Clemens Eppner, Bryan Peele, Byron Boots, Dieter Fox
Conference on Robot Learning (CoRL), 2022
"This is my unicorn, Fluffy": Personalizing frozen vision-language representations
Niv Cohen, Rinon Gal, Eli Meirom, Gal Chechik, Yuval Atzmon
ECCV 2022
Structural Pruning via Latency-Saliency Knapsack
Maying Shen, Hongxu Danny Yin, Pavlo Molchanov, Lei Mao, Jianna Liu, Jose M. Alvarez
NeurIPS 2022
SPoVT: Semantic-Prototype Variational Transformer for Dense Point Cloud Semantic Completion
Sheng-Yu Huang, Hao-Yu Hsu, Yu-Chiang Frank Wang
NeurIPS 2022
Paraphrasing Is All You Need for Novel Object Captioning
Cheng-Fu Yang, Yao-Hung Hubert Tsai, Wan-Cyuan Fan, Ruslan Salakhutdinov, Louis-Philippe Morency, Frank Wang
NeurIPS 2022
Embodied Scene-aware Human Pose Estimation
Zhengyi Luo, Shun Iwase, Ye Yuan, Kris Kitani
NeurIPS 2022
6-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmark
Stephen Tyree, Jonathan Tremblay, Stan Birchfield, et al.
IROS 2022
Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty
Boris Ivanovic, Kuan-Hui Lee, Pavel Tokmakov, Blake Wulfe, Adrien Gaidon, Marco Pavone
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2022
AdvDO: Realistic Adversarial Attacks for Trajectory Prediction
Yulong Cao, Chaowei Xiao, Anima Anandkumar, Danfei Xu, Marco Pavone
European Conference on Computer Vision (ECCV) 2022
Text2LIVE: Text-Driven Layered Image and Video Editing
Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kasten, Tali Dekel
ECCV 2022
LANA: Latency Aware Network Acceleration
Pavlo Molchanov, Jimmy Hall, Hongxu Danny Yin, Jan Kautz, Nicolo Fusi, Arash Vahdat
European Conference on Computer Vision (ECCV), 2022
Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising
Jon Hasselgren, Nikolai Hofmann, Jacob Munkberg
NeurIPS 2022
Audio-Visual Segmentation
Jinxin Zhou, Yiran Zhong, Stan Birchfield, et al.
ECCV 2022
Variable Bitrate Neural Fields
Towaki Takikawa, Alex Evans, Jonathan Tremblay, Thomas Müller, Morgan McGuire, Alec Jacobson, Sanja Fidler
ACM SIGGRAPH 2022 Conference Proceedings
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, Daniel Cohen-Or
ICLR 2023
Top 25%
Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
Thomas Müller, Alex Evans, Christoph Schied, Alex Keller
ACM Transactions on Graphics (SIGGRAPH 2022)
Best Technical Paper, SIGGRAPH 2022, THE BEST INVENTIONS OF 2022, TIME
CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs
Jiteng Mu, Shalini De Mello, Zhiding Yu, Nuno Vasconcelos, Xiaolong Wang, Sifei Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Whose Track Is It Anyway? Improving Robustness to Tracking Errors with Affinity-Based Prediction
Xinshuo Weng, Boris Ivanovic, Kris Kitani, Marco Pavone
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning
Yuxiao Chen, Boris Ivanovic, Marco Pavone
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Ifor: Iterative flow minimization for robotic object rearrangement
Ankit Goyal, Arsalan Mousavian, Chris Paxton, Yu-Wei Chao, Brian Okorn, Jia Deng, Dieter Fox
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Efficient Geometry-aware 3D Generative Adversarial Networks
Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Oral presentation
GroupViT: Semantic Segmentation Emerges from Text Supervision
Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, Xiaolong Wang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Ye Yuan, Umar Iqbal, Pavlo Molchanov, Kris Kitani, Jan Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022 (Ora…
FreeSOLO: Learning to Segment Objects without Annotations
Xinlong Wang, Zhiding Yu, Shalini De Mello, Jan Kautz, Anima Anandkumar, Chunhua Shen, Jose M. Alvarez
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Injecting Planning-Awareness into Prediction and Detection Evaluation
Boris Ivanovic, Marco Pavone
IEEE Intelligent Vehicles Symposium (IV) 2022
MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation
Xinshuo Weng, Boris Ivanovic, Marco Pavone
IEEE Intelligent Vehicles Symposium (IV) 2022
Interaction-Dynamics-Aware Perception Zones for Obstacle Detection Safety Evaluation
Sever Topan, Karen Leung, Yuxiao Chen, Pritish Tupekar, Edward Schmerling, Jonas Nilsson, Michael Cox, Marco Pavone
IEEE Intelligent Vehicles Symposium (IV) 2022
A-ViT: Adaptive Tokens for Efficient Vision Transformer
Hongxu Danny Yin, Arash Vahdat, Jose M. Alvarez, Arun Mallya, Jan Kautz, Pavlo Molchanov
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022 (Ora…
When to Prune? A Policy towards Early Structural Pruning
Maying Shen, Pavlo Molchanov, Hongxu Danny Yin, Jose M. Alvarez
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022
GradViT: Gradient Inversion of Vision Transformers
Ali Hatamizadeh, Hongxu Danny Yin, Holger Roth, Wenqi Li, Jan Kautz, Daguang Xu, Pavlo Molchanov
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers
Yu-Wei Chao, Chris Paxton, Yu Xiang, Wei Yang, Balakumar Sundaralingam, Tao Chen, Adithya Murali, Maya Cakmak, Dieter Fox
IEEE International Conference on Robotics and Automation (ICRA) 2022
Propagating State Uncertainty Through Trajectory Forecasting
Boris Ivanovic, Yifeng Lin, Shubham Shrivastava, Punarjay Chakravarty, Marco Pavone
IEEE International Conference on Robotics and Automation (ICRA) 2022
A Dataset and Explorer for 3D Signed Distance Functions
Towaki Takikawa, Andrew Glassner, Morgan McGuire
Journal of Computer Graphics Techniques
Neural Fields in Visual Computing and Beyond
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, Srinath Sridhar
Computer Graphics Forum (Eurographics 2022)
Learning Continuous Environment Fields via Implicit Functions
Xueting Li, Shalini De Mello, Xiaolong Wang, Ming-Hsuan Yang, Jan Kautz, Sifei Liu
International Conference on Learning Representations (ICLR), 2022
Efficient Token Mixing for Transformers via Adaptive Fourier Neural Operators
"This is my unicorn, Fluffy": Personalizing frozen vision-language representations
Niv Cohen, Rinon Gal, Gal Chechik, Yuval Atzmon
ECCV 2022
Oral
Learning Contrastive Representation for Semantic Correspondence
Taihong Xiao, Sifei Liu, Shalini De Mello, Zhiding Yu, Jan Kautz, Ming-Hsuan Yang
International Journal of Computer Vision (IJCV)
PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation
Alexey Kamenev, Lirui Wang, Ollin Boer Bohan, Ishwar Kulkarni, Bilal Kartal, Artem Molchanov, Stan Birchfield, David Nister, Nikolai Smolyanskiy
ICRA 2022
Score-Based Generative Modeling with Critically-Damped Langevin Diffusion
Tim Dockhorn, Arash Vahdat, Karsten Kreis
International Conference on Learning Representations (ICLR), 2022 (Spotlight)
Tackling the Generative Learning Trilemma with Denoising Diffusion GANs
Zhisheng Xiao, Karsten Kreis, Arash Vahdat
International Conference on Learning Representations (ICLR) 2022 (Spotlight)
Single-Stage Keypoint-Based Category-Level Object Pose Estimation from an RGB Image
Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patricio A. Vela, Stan Birchfield
ICRA 2022
Keypoint-Based Category-Level Object Pose Tracking from an RGB Sequence with Uncertainty Estimation
Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patricio A. Vela, Stan Birchfield
ICRA 2022
Displacement-Invariant Cost Computation for Efficient Stereo Matching
Yiran Zhong, Charles Loop, Wonmin Byeon, Stan Birchfield, et al.
IJCV
RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis
Jonathan Tremblay, Moustafa Meshry, Stan Birchfield, Alex Evans, Jan Kautz, Alex Keller, Sameh Khamis, Charles Loop, Nate Morrical, Thomas Müller, Koki Nagano, Towaki Takikawa, Stan Birchfield
ECCV 2022 Workshop on Learning to Generate 3D Shapes and Scenes

2021

KAMA: 3D Keypoint Aware Body Mesh Articulation
Umar Iqbal, Kevin Xie, Kelly Guo, Jan Kautz, Pavlo Molchanov
International Conference on 3D Vision
Standard vs. Learning-based Codecs for Real Time Endoscopic Video Transmission
Iuri Frosio, Aldo Marzullo, Martina Golini, Elena De Momi, Michele Catellani, Francesco Calimeri, Giuseppe Fiameni
AIABI-2021 Italian Workshop on Artificial Intelligence and Applications for Bus…
Extracting Triangular 3D Models, Materials, and Lighting From Images
Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, Sanja Fidler
CVPR 2022 (Oral)
Noise-Aware Video Saliency Prediction
Ekta Prashnani, Orazio Gallo, Joohwan Kim, Josef Spjut, Pradeep Sen, Iuri Frosio
The British Machine Vision Conference (BMVC) - 2021
Don’t Generate Me: Training Differentially Private Generative Models with Sinkhorn Divergence
Tianshi Cao, Alex Bie, Arash Vahdat, Sanja Fidler, Karsten Kreis
Neural Information Processing Systems (NeurIPS) 2021
Score-based Generative Modeling in Latent Space
Arash Vahdat, Karsten Kreis, Jan Kautz
Neural Information Processing Systems (NeurIPS) 2021
A Contrastive Learning Approach for Training Variational Autoencoder Priors
Jyoti Aneja, Alexander Schwing, Jan Kautz, Arash Vahdat
Neural Information Processing Systems (NeurIPS) 2021
Controllable and Compositional Generation with Latent-Space Energy-Based Models
Weili Nie, Arash Vahdat, Anima Anandkumar
Neural Information Processing Systems (NeurIPS) 2021
Self-Supervised Object Detection via Generative Image Synthesis
Siva Karthik Mustikovela, Shalini De Mello, Aayush Prakash, Umar Iqbal, Sifei Liu, Thu Nguyen-Phuoc, Carsten Rother, Jan Kautz
International Conference on Computer Vision (ICCV) 2021
Self-Supervised Real-to-Sim Scene Generation
Aayush Prakash, Shoubhik Debnath, Jean-Francois Lafleche, Eric Cameracci, Gavriel State, Stan Birchfield, Marc T. Law
ICCV 2021
Multi-View Fusion for Multi-Level Robotic Scene Understanding
Yunzhi Lin, Jonathan Tremblay, Stephen Tyree, Patricio A. Vela, Stan Birchfield
IROS 2021
Learning to Track Instances without Video Annotations
Yang Fu, Sifei Liu, Umar Iqbal, Shalini De Mello, Humphrey Shi, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Oral presentation
Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes
Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, Sanja Fidler
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Binary TTC: A Temporal Geofence for Autonomous Navigation
Abhishek Badki, Orazio Gallo, Jan Kautz, Pradeep Sen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Best Student Paper Honorable Mention, CVPR 2021
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing
Ting-Chun Wang, Arun Mallya, Ming-Yu Liu
CVPR
Weakly-Supervised Physically Unconstrained Gaze Estimation
Rakshit Kothari, Shalini De Mello, Umar Iqbal, Wonmin Byeon, Seonwook Park, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Oral presentation
Deep Two-View Structure-from-Motion Revisited
Jianyuan Wang, Yiran Zhong, Yuchao Dai, Stan Birchfield, Kaihao Zhang, Nikolai Smolyanskiy, Hongdong Li
CVPR 2021
See through Gradients: Image Batch Recovery via GradInversion
Hongxu Danny Yin, Arun Mallya, Arash Vahdat, Jose M. Alvarez, Jan Kautz, Pavlo Molchanov
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Optimal Quantization Using Scaled Codebook
Yerlan Idelbayev, Pavlo Molchanov, Maying Shen, Hongxu Danny Yin, Miguel A. Carreira-Perpinan, Jose M. Alvarez
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021
DexYCB: A Benchmark for Capturing Hand Grasping of Objects
Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj Narang, Karl Van Wyk, Umar Iqbal, Stan Birchfield, Jan Kautz, Dieter Fox
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
Zhisheng Xiao, Karsten Kreis, Jan Kautz, Arash Vahdat
International Conference on Learning Representations (ICLR), 2021 (Spotlight)
NViSII: A Scriptable Tool for Photorealistic Image Generation
Nathan Morrical, Jonathan Tremblay, Yunzhi Lin, Stephen Tyree, Stan Birchfield, Valerio Pascucci, Ingo Wald
SDG Workshop at ICLR 2021
RGB-D Local Implicit Function for Depth Completion of Transparent Objects
Luyang Zhu, Arsalan Mousavian, Yu Xiang, Hammad Mazhar, Jozef van Eenbergen, Shoubhik Debnath, Dieter Fox
CVPR 2021
Reactive Human-to-Robot Handovers of Arbitrary Objects
Wei Yang, Chris Paxton, Arsalan Mousavian, Yu-Wei Chao, Maya Cakmak, Dieter Fox
ICRA 2021
Best Paper in Human-Robot Interaction, ICRA 2021
Robust Vision-Based Cheat Detection in Competitive Gaming
Aditya Jonnalagadda, Iuri Frosio, Seth Schenider, Morgan McGuire, Joohwan Kim
I3D ’21
Self-Supervised Learning for Domain Adaptation on Point-Clouds
Idan Achituve, Haggai Maron, Gal Chechik
Winter Conference on Applications of Computer Vision (WACV), 2021
From Generalized Zero-Shot Learning to Long-Tail with Class Descriptors
Dvir Samuel, Yuval Atzmon, Gal Chechik
Winter Conference on Applications of Computer Vision (WACV) 2021
Data-Free Knowledge Distillation for Object Detection
Akshay Chawla, Hongxu Danny Yin, Pavlo Molchanov, Jose M. Alvarez
WACV 2021

2020

Online Adaptation for Consistent Mesh Reconstruction in the Wild
Xueting Li, Sifei Liu, Shalini De Mello, Kihwan Kim, Xiaolong Wang, Ming-Hsuan Yang, Jan Kautz
Neural Information Processing Systems (NeurIPS) 2020
Self-Learning Transformations for Improving Gaze and Head Redirection
Yufeng Zheng, Seonwook Park, Xucong Zhang, Shalini De Mello, Otmar Hilliges
Neural Information Processing Systems (NeurIPS) 2020
A Causal View of Compositional Zero-Shot Recognition
Yuval Atzmon, Felix Kreuk, Uri Shalit, Gal Chechik
Neural Information Processing Systems (NeurIPS) 2020 (Spotlight)
Generative View Synthesis: From Single-View Semantics to Novel-View Images
Tewodros Abtegebrial, Varun Jampani, Orazio Gallo, Didier Stricker
NeurIPS
Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning
Weili Nie, Zhiding Yu, Lei Mao, Ankit B. Patel, Yuke Zhu, Anima Anandkumar
Conference on Neural Information Processing Systems (NeurIPS) 2020 (Spotlight)
Neural Networks with Recurrent Generative Feedback
Yujia Huang, James Gornet, Sihui Dai, Zhiding Yu, Tan Nguyen, Doris Y. Tsao, Anima Anandkumar
Conference on Neural Information Processing Systems (NeurIPS) 2020
Learning Deformable Tetrahedral Meshes for 3D Reconstruction
Jun Gao, Wenzheng Chen, Tommy Xiang, Clement Fuji Tsang, Alec Jacobson, Morgan McGuire
NeurIPS
Neural FFTs for Universal Texture Image Synthesis
Morteza Mardani, Guilin Liu, Aysegul Dundar, Shiqiu Liu, Andrew Tao, Bryan Catanzaro
NeurIPS 2020
ZEST: Zero-shot Learning from Text Descriptions using Textual Similarity and Visual Summarization
Tzuf Paz-Argaman, Yuval Atzmon, Gal Chechik, Reut Tsarfaty
Findings of EMNLP
Learning Object Permanence from Video
Aviv Shamsian, Ofri Kleinfeld, Amir Globerson, Gal Chechik
ECCV 2020
LAMP: Large Deep Nets with Automated Model Parallelism for Image Segmentation
Wentao Zhu, Can Zhao, Wenqi Li, Holger Roth, Ziyue Xu, Daguang Xu
MICCAI 2020
UNAS: Differentiable Architecture Search Meets Reinforcement Learning
Arash Vahdat, Arun Mallya, Ming-Yu Liu, Jan Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021
UFO2: A Unified Framework towards Omni-supervised Object Detection
Zhongzheng Ren, Zhiding Yu, Xiaodong Yang, Ming-Yu Liu, Alexander G. Schwing, Jan Kautz
European Conference on Computer Vision (ECCV) 2020
Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification
Yang Zou, Xiaodong Yang, Zhiding Yu, B. V. K. Vijaya Kumar, Jan Kautz
European Conference on Computer Vision (ECCV) 2020 (Oral)
World-Consistent Video-to-Video Synthesis
Arun Mallya, Ting-Chun Wang, Karan Sapra, Ming-Yu Liu
ECCV
Self-supervised Single-view 3D Reconstruction via Semantic Consistency
Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Ming-Hsuan Yang, Jan Kautz
European Conference on Computer Vision (ECCV) 2020
Weakly-Supervised 3D Hand Pose Estimation via Biomechanical Constraints
Adrian Spurr, Umar Iqbal, Pavlo Molchanov, Otmar Hilliges, Jan Kautz
European Conference on Computer Vision, 2020
Indirect Object-to-Robot Pose Estimation from an External Monocular RGB Camera
Jonathan Tremblay, Stephen Tyree, Terry Mosier, Stan Birchfield
IROS 2020
Weakly Supervised One-stage Vision and Language Disease Detection using Large Scale Pneumonia and Pneumothorax Studies
Leo Tam, Xiaosong Wang, Evrim Turkbey, Kevin Lu, Yuhong Wen, Daguang Xu
MICCAI 2020
Automated Synthetic-to-Real Generalization
Wuyang Chen, Zhiding Yu, Zhangyang Wang, Anima Anandkumar
International Conference on Machine Learning (ICML) 2020
Semi-Supervised StyleGAN for Disentanglement Learning
Weili Nie, Tero Karras, Animesh Garg, Shoubhik Debnath, Anjul Patney, Ankit B. Patel, Anima Anandkumar
International Conference on Machine Learning (ICML) 2020
Angular Visual Hardness
Beidi Chen, Weiyang Liu, Zhiding Yu, Jan Kautz, Anshumali Shrivastava, Animesh Garg, Anima Anandkumar
International Conference on Machine Learning (ICML) 2020
NVAE: A Deep Hierarchical Variational Autoencoder
Arash Vahdat, Jan Kautz
Neural Information Processing Systems (NeurIPS) 2020 (Spotlight)
Contrastive Learning for Weakly Supervised Phrase Grounding
Tanmay Gupta, Arash Vahdat, Gal Chechik, Xiaodong Yang, Jan Kautz, Derek Hoiem
European Conference on Computer Vision (ECCV) 2020 (Spotlight)
Instance-aware, Context-focused, and Memory-efficient Weakly Supervised Object Detection
Zhongzheng Ren, Zhiding Yu, Xiaodong Yang, Ming-Yu Liu, Yong Jae Lee, Alexander G. Schwing, Jan Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Regularizing Neural Networks via Minimizing Hyperspherical Energy
Weiyang Liu, Rongmei Lin, Zhen Liu, Chen Feng, Zhiding Yu, James M. Rehg, Li Xiong, Le Song
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Bi3D: Stereo Depth Estimation via Binary Classifications
Abhishek Badki, Alejandro Troccoli, Kihwan Kim, Jan Kautz, Pradeep Sen, Orazio Gallo
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Meshlet Priors for 3D Mesh Reconstruction
Abhishek Badki, Orazio Gallo, Jan Kautz, Pradeep Sen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
Hongxu Danny Yin, Pavlo Molchanov, Jose M. Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K. Jha, Jan Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020 (Ora…
Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths
Jae shin Yoon, Kihwan Kim, Orazio Gallo, Hyunsoo Park, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Two-shot Spatially-varying BRDF and Shape Estimation
Mark Boss, Varun Jampani, Kihwan Kim, Hendrik P.A. Lensch, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Self-Supervised Viewpoint Learning From Image Collections
Siva Karthik Mustikovela, Varun Jampani, Shalini De Mello, Sifei Liu, Umar Iqbal, Carsten Rother, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020
MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views
Ke Chen, Ryan, Nikolai, Stan Birchfield, Alexander Popov, David Wehr, Ibrahim Eden, Joachim Pehserl
IROS 2020
Learning Canonical Representations for Scene Graph to Image Generation
Roei Herzig, Amir Bar, Huijuan Xu, Gal Chechik, Trevor Darrell, Amir Globerson
ECCV 2020
Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild
Umar Iqbal, Pavlo Molchanov, Jan Kautz
IEEE Computer Vision and Pattern Recognition
6-DOF Grasping for Target-driven Object Manipulation in Clutter
Adithyavairavan Murali, Arsalan Mousavian, Clemens Eppner, Chris Paxton, Dieter Fox
ICRA 2020
Best Paper Finalist in Robot Manipulation, ICRA 2020, Best Student Paper Finalist, ICRA 2020
Self-supervised 6D Object Pose Estimation for Robot Manipulation
Xinke Deng, Yu Xiang, Arsalan Mousavian, Clemens Eppner, Timothy Bretl, Dieter Fox
2020 IEEE International Conference on Robotics and Automation (ICRA)
Camera-to-Robot Pose Estimation from a Single Image
Timothy E. Lee, Jonathan Tremblay, Thang To, Jia Cheng, Terry Mosier, Oliver Kroemer, Dieter Fox, Stan Birchfield
ICRA 2020
Toward Sim-to-Real Directional Semantic Grasping
Shariq Iqbal, Jonathan Tremblay, Thang To, Jia Cheng, Erik Leitch, Andy Campbell, Kirby Leung, Duncan McKay, Stan Birchfield
ICRA 2020
How to close sim-real gap? transfer with segmentation!
Mengyuan Yan, Qingyun Sun, Iuri Frosio, Stephen Tyree, Jan Kautz
arxiv
DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System
Ankur Handa, Karl Van Wyk, Wei Yang, Jacky Liang, Yu-Wei Chao, Qian Wan, Stan Birchfield, Nathan Ratliff, Dieter Fox
ICRA 2020
NRMVS: Non-Rigid Multi-view Stereo
Matthias Innmann, Kihwan Kim, Jinwei Gu, Matthias Niessner , Charles Loop, Marc Stamminger, Jan Kautz
IEEE Winter Conference on Applications of Computer Vision (WACV ’20)
SymGAN: Orientation Estimation without Annotation for Symmetric Objects
Phil Ammirato, Jonathan Tremblay, Ming-Yu Liu, Alexander Berg, Dieter Fox
WACV
Neurreg: Neural registration and its application to image segmentation
Wentao Zhu, Andriy Myronenko, Ziyue Xu, Wenqi Li, Holger Roth, Yufang Huang, Fausto Milletari, Daguang Xu
WACV
Domain Stylization: A Fast Covariance Matching Framework towards Domain Adaptation
Aysegul Dundar, Ming-Yu Liu, Zhiding Yu, Ting-Chun Wang, John Zedlewski, Jan Kautz
IEEE Transactions on Pattern Analysis and Machine Intelligence
Displacement-Invariant Cost Computation for Efficient Stereo Matching
Yiran Zhong, Charles Loop, Wonmin Byeon, Stan Birchfield, Yuchao Dai, Kaihao Zhang, Alexey Kamenev, Thomas Breuel, Hongdong Li, Jan Kautz
arXiv
Improving Deep Stereo Network Generalization with Geometric Priors
Jialiang Wang, Varun Jampani, Deqing Sun, Charles Loop, Stan Birchfield, Jan Kautz
arXiv

2019

Few-Shot Video-to-Video Synthesis
Ting-Chun Wang, Ming-Yu Liu, Andrew Tao, Guilin Liu, Jan Kautz, Bryan Catanzaro
NeurIPS
Joint-task Self-supervised Learning for Temporal Correspondence
Xueting Li, Sifei Liu, Shalini De Mello, Xiaolong Wang, Jan Kautz, Ming-Hsuan Yang
Neural Information Processing Systems (NeurIPS) 2019
Dance to Music
Hsin-Ying Lee, Xiaodong Yang, Ming-Yu Liu, Ting-Chun Wang, Yu-Ding Lu, Ming-Hsuan Yang, Jan Kautz
NeurIPS
Joint Optimization for Cooperative Image Captioning
Gilad Vered, Gal Oren, Yuval Atzmon, Gal Chechik
International conference on computer vision (ICCV)
Content-Consistent Generation of Realistic Eyes with Style
Marcel Bühler , Seonwook Park, Shalini De Mello, Xucong, Otmar Hilliges
International Conference on Computer Vision Workshop (ICCVW) 2019
Winner (1st place) Synthetic Eye Generation Challenge
Neural Inverse Rendering of an Indoor Scene from a Single Image
Soumyadip Sengupta, Jinwei Gu, Kihwan Kim, Guilin Liu, David W. Jacobs, Jan Kautz
IEEE International Conference on Computer Vision (ICCV 2019)
Few-Shot Adaptive Gaze Estimation
Seonwook Park, Shalini De Mello, Pavlo Molchanov, Umar Iqbal, Otmar Hilliges, Jan Kautz
International Conference on Computer Vision (ICCV) 2019
Oral presentation
PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data
Zheng Tang, Milind Naphade, Stan Birchfield, Jonathan Tremblay, William Hodge, Ratnesh Kumar, Shuo Wong, Xiaodong Yang
ICCV 2019
SENSE: A Shared Encoder Network for Scene-flow Estimation
Huaizu Jiang, Deqing Sun, Varun Jampani, Zhaoyang Lv, Erik Learned-Miller, Jan Kautz
International Conference in Computer Vision
Extreme View Synthesis
Inchang Choi, Orazio Gallo, Alejandro Troccoli, Min H. Kim, Jan Kautz
IEEE International Conference of Computer Vision
PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, Bharath Hariharan
ICCV
6-DOF GraspNet: Variational Grasp Generation for Object Manipulation
Arsalan Mousavian, Clemens Eppner, Dieter Fox
ICCV 2019
Neural Turtle Graphics for Modeling City Road Layouts
Hang Chu, Daiqing Li, David Acuna, Amlan Kar, Maria Shugrina, Xinkai Wei, Ming-Yu Liu, Antonio Torralba, Sanja Fidler
ICCV
Few-Shot Unsupervised Image-to-Image Translation
Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, Jan Kautz
ICCV
Meta-Sim: Learning to Generate Synthetic Datasets
Amlan Kar, Aayush Prakash, Ming-Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, Sanja Fidler
ICCV
Confidence Regularized Self-Training
Yang Zou, Zhiding Yu, Xiaofeng Liu, B. V. K. Vijaya Kumar, Jinsong Wang
IEEE/CVF International Conference on Computer Vision (ICCV) 2019 (Oral)
Learning Propagation for Arbitrarily-Structured Data
Sifei Liu, Xueting Li, Varun Jampani, Shalini De Mello, Jan Kautz
International Conference on Computer Vision (ICCV) 2019
Few-Shot Viewpoint Estimation
Hung-Yu Tseng, Shalini De Mello, Jonathan Tremblay, Sifei Liu, Stan Birchfield, Ming-Hsuan Yang, Jan Kautz
British Machine Vision Conference (BMVC) 2019
Video Stitching for Linear Camera Arrays
Wei-Sheng Lai, Orazio Gallo, Jinwei Gu, Deqing Sun, Ming-Hsuan Yang, Jan Kautz
British Machine Vision Conference
SIDOD: A Synthetic Image Dataset for 3D Object Pose Recognition with Distractors
Mona Jalal, Josef Spjut, Ben Boudaoud, Margrit Betke
WiCV
Semantic Image Synthesis with Spatially-Adaptive Normalization
Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu
CVPR
Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera
Chao Liu, Jinwei Gu, Kihwan Kim, Srinivasa Narasimhan, Jan Kautz
IEEE CVPR 2019 (Oral)
Joint Discriminative and Generative Learning for Person Re-identification
Zhedong Zheng, Xiaodong Yang, Zhiding Yu, Liang Zheng, Yi Yang, Jan Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2019
Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments
Xueting Li, Sifei Liu, Kihwan Kim, Xiaolong Wang, Ming-Hsuan Yang, Jan Kautz
CVPR 2019
STEP: Spatio-Temporal Progressive Learning for Video Action Detection
Xitong Yang, Xiaodong Yang, Ming-Yu Liu, Fanyi Xiao, Larry Davis, Jan Kautz
CVPR
PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image
Chen Liu, Kihwan Kim, Jinwei Gu, Yasutaka Furukawa, Jan Kautz
IEEE CVPR 2019 (Oral)
Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation
Anurag Ranjan, Varun Jampani, Lukas Balles, Kihwan Kim, Deqing Sun, Jonas Wulff, Michael J. Black
IEEE CVPR 2019
Pixel-Adaptive Convolutional Neural Networks
Hang Su, Varun Jampani, Deqing Sun, Orazio Gallo, Erik-Learned Miller, Jan Kautz
Computer Vision and Pattern Recognition (CVPR), 2019
CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification
Zheng Tang, Milind Naphade, Ming-Yu Liu, Xiaodong Yang, Stan Birchfield, Shuo Wang, Ratnesh Kumar, David Anastasiu, Jenq-Neng Hwan
CVPR 2019
SCOPS: Self-Supervised Co-Part Segmentation
Wei-Chih Hung, Varun Jampani, Sifei Liu, Pavlo Molchanov, Ming-Hsuan Yang , Jan Kautz
CVPR 2019
Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data
Aayush Prakash, Shaad Boochoon, Mark Brophy, David Acuna, Eric Cameracci, Gavriel State, Omer Shapira, Stan Birchfield
ICRA 2019
Learning Linear Transformations for Fast Image and Video Style Transfer
Xueting Li, Sifei Liu, Ming-Hsuan Yang, Jan Kautz
CVPR 2019
Informative Object Annotations: Tell Me Something I Don't Know
Lior Bracha, Gal Chechik
Computer Vision and Pattern Recognition
Adaptive Confidence Smoothing for Generalized Zero-Shot Learning
Yuval Atzmon, Gal Chechik
Computer Vision and Pattern Recognition (CVPR) 2019
Unsupervised Stylish Image Description Generation via Domain Layer Norm
Cheng-Kuan Chen, Zhu-Feng Pan, Ming-Yu Liu, Min Sun
AAAI
Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation
Deqing Sun, Xiaodong Yang, Ming-Yu Liu, Jan Kautz
TPAMI
A Fusion Approach for Multi-Frame Optical Flow Estimation
Zhile Ren, Orazio Gallo, Deqing Sun, Ming-Hsuan Yang, Erik B. Sudderth, Jan Kautz
IEEE Winter conference of Applications of Computer Vision (WACV)

2018

Localization-Aware Active Learning for Object Detection
Chieh-Chi Kao, Teng-Yok Lee, Pradeep Sen, Ming-Yu Liu
ACCV
Video-to-Video Synthesis
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, Bryan Catanzaro
NIPS
Context-aware Synthesis and Placement of Object Instances
Donghoon Lee, Sifei Liu, Jinwei Gu, Ming-Yu Liu, Ming-Hsuan Yang, Jan Kautz
NIPS
Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction
Roei Herzig, Moshiko Raboh, Gal Chechik, Jonathan Berant, Amir Globerson
Neural Information processing systems (NeurIPS)
Learning towards Minimum Hyperspherical Energy
Weiyang Liu, Rongmei Lin, Zhen Liu, Lixin Liu, Zhiding Yu, Bo Dai, Le Song
Conference on Neural Information Processing Systems (NeurIPS) 2018
Context-aware Synthesis and Placement of Object Instances
Donghoon Lee, Sifei Liu, Jinwei Gu, Ming-Yu Liu, Ming-Hsuan Yang, Jan Kautz
NIPS
Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects
Jonathan Tremblay, Thang To, Bala Sundaralingam, Yu Xiang, Dieter Fox, Stan Birchfield
Conference on Robot Learning (CoRL) 2018
Hand Pose Estimation via Latent 2.5 D Heatmap Regression
Umar Iqbal, Pavlo Molchanov, Thomas Breuel, Juergen Gall, Jan Kautz
ECCV2018
Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset
Qi Guo, Iuri Frosio, Orazio Gallo, Todd Zickler, Jan Kautz
ECCV 2018
Separating Reflection and Transmission Images in the Wild
Patrick Wieschollek, Orazio Gallo, Jinwei Gu, Jan Kautz
European Conference of Computer Vision (ECCV)
Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation
Zhaoyang Lv, Kihwan Kim, Alejandro Troccoli, Deqing Sun, James M. Rehg, Jan Kautz
European Conference on Computer Vision (ECCV 2018)
Simultaneous Edge Alignment and Learning
Zhiding Yu, Weiyang Liu, Yang Zou, Chen Feng, Srikumar Ramalingam, B. V. K. Vijaya Kumar, Jan Kautz
European Conference on Computer Vision (ECCV) 2018
Switchable Temporal Propagation Network
Sifei Liu, Guangyu Zhong, Shalini De Mello, Jinwei Gu, Varun Jampani
European Conference on Computer Vision (ECCV) 2018
Image Inpainting for Irregular Holes Using Partial Convolutions
Guilin Liu, Fitsum A. Reda, Kevin Shih, Ting-Chun Wang, Andrew Tao, Bryan Catanzaro
ECCV
Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training
Yang Zou, Zhiding Yu, B. V. K. Vijaya Kumar, Jinsong Wang
European Conference on Computer Vision (ECCV) 2018
HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration
Ben Eckart, Kihwan Kim, Jan Kautz
European Conference on Computer Vision (ECCV 2018)
Multimodal Unsupervised Image-to-Image Translation
Xun Huang, Ming-Yu Liu, Serge Belongie, Jan Kautz
ECCV
A Closed-form Solution to Photorealistic Image Stylization
Yijun Li, Ming-Yu Liu, Xueting Li, Ming-Hsuan Yang, Jan Kautz
ECCV
EOE: Expected Overlap Estimation over Unstructured Point Cloud Data
Ben Eckart, Kihwan Kim, Jan Kautz
International Conference on 3D Vision (3DV) 2018
3D MRI Brain Tumor Segmentation Using Autoencoder Regularization
Andriy Myronenko
MICCAI, BrainLes, 2018
Superpixel Sampling Networks
Varun Jampani, Deqing Sun, Ming-Yu Liu, Ming-Hsuan Yang, Jan Kautz
European Conference on Computer Vision (ECCV), 2018
Noise2Noise: Learning Image Restoration without Clean Data
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, Timo Aila
Proc. ICML 2018
Light-weight Head Pose Invariant Gaze Tracking
Rajeev Ranjan, Shalini De Mello, Jan Kautz
IEEE Computer Vision and Pattern Recognition Workshop (CVPRW) 2018
Best Paper (runner up) Workshop on Analysis and Modeling of Faces and Gestures
Learning Superpixels with Segmentation-Aware Affinity Losse
Wei-Chih Tu, Ming-Yu Liu, Varun Jampani, Deqing Sun, Shao-Yi Chien, Ming-Hsuan Yang, Jan Kautz
CVPR
MoCoGAN: Decomposing Motion and Content for Video Generation
Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, Jan Kautz
CVPR
Improving Landmark Localization with Semi-Supervised Learning
Sina Honari, Pavlo Molchanov, Stephen Tyree, Pascal Vincent, Christopher Pal, Jan Kautz
CVPR
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz
CVPR 2018
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
Shanxin Yuan, Guillermo Garcia-Hernando, Bjorn Stenger, Pavlo Molchanov, Jan Kautz, Sina Honari
Conference on Computer Vision and Pattern Recognition
Decoupled Networks
Weiyang Liu, Zhen Liu, Zhiding Yu, Bo Dai, Rongmei Lin, Yisen Wang, James M. Rehg, Le Song
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Deep Semantic Face Deblurring
Ziyi Shen, Wei-Sheng Lai, Tingfa Xu, Jan Kautz, Ming-Hsuan Yang
IEEE Computer Vision and Pattern Recognition (CVPR)
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, Bryan Catanzaro
CVPR
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Deqing Sun, Xiaodong Yang, Ming-Yu Liu, Jan Kautz
CVPR
SPLATNet: Sparse Lattice Networks for Point Cloud Processing
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz
CVPR 2018 (oral)
Learning Strict Identity Mappings in Deep Residual Networks
Xin Yu, Zhiding Yu, Srikumar Ramalingam
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Making Convolutional Networks Recurrent for Visual Sequence Learning
Xiaodong Yang, Pavlo Molchanov, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Geometry-Aware Learning of Maps for Camera Localization
Samarth Brahmbhatt, Jinwei Gu, Kihwan Kim, James Hays, Jan Kautz
CVPR 2018 (Spotlight)
Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation
Jonathan Tremblay, Thang To, Stan Birchfield
CVPR 2018 Workshop on Real World Challenges and New Benchmarks for Deep Learnin…
PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, Dieter Fox
Robotics: Science and Systems (RSS)
Synthetically Trained Neural Networks for Learning Human-Readable Plans from Real-World Demonstrations
Jonathan Tremblay, Thang To, Artem Molchanov, Stephen Tyree, Jan Kautz, Stan Birchfield
IEEE International Conference on Robotics and Automation (ICRA) 2018
Probabilistic AND-OR Attribute Grouping for Zero-Shot Learning
Yuval Atzmon, Gal Chechik
The conference on uncertainty in artificial intelligence (UAI 2018)
Reblur2Deblur: Deblurring Videos via Self-Supervised Learning
Huaijin Chen, Jinwei Gu, Orazio Gallo, Ming-Yu Liu, Ashok Veeraraghavan, Jan Kautz
IEEE International Conference on Computational Photography (ICCP)
Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization
Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, Stan Birchfield
CVPR 2018 Workshop on Autonomous Driving
IamNN: Iterative and Adaptive Mobile Neural Network for Efficient Image Classification
Sam Leroux, Pavlo Molchanov, Pieter Simoens, Bart Dhoedt, Thomas Breuel, Jan Kautz
International Conference on Learning Representations, Workshop
On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach
Nikolai Smolyanskiy, Alexey Kamenev, Stan Birchfield
CVPR 2018 Workshop on Autonomous Driving

2017

Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Control
Mengyuan Yan, Iuri Frosio, Stephen Tyree, Jan Kautz
NIPS 2017 Workshop on Acting and Interacting in the Real World: Challenges in …
On Nearest Neighbors in Non Local Means Denoising
Iuri Frosio, Jan Kautz
Neural Information Processing Systems (NIPS) 2017 Workshop on Nearest Neighbors…
Unsupervised Image-to-Image Translation Networks
Ming-Yu Liu, Thomas Breuel, Jan Kautz
NIPS
Learning Affinity via Spatial Propagation Networks
Sifei Liu, Shalini De Mello, Jinwei Gu, Guangyu Zhong, Ming-Hsuan Yang, Jan Kautz
Conference on Neural Information Processing Systems (NIPS) 2017
Learning to Super-Resolve Blurry Face and Text Images
Xiangyu Xu, Deqing Sun, Jinshan Pan, Yujin Zhang, Hanspeter Pfister, Ming-Hsuan Yang
International Conference on Computer Vision
Semantic Video CNNs through Representation Warping
Raghudeep Gadde, Varun Jampani, Peter V. Gehler
International Conference on Computer Vision (ICCV'17)
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
Robert Maier, Kihwan Kim, Daniel Cremers, Jan Kautz, Matthias Nießner
IEEE International Conference on Computer Vision (ICCV 2017)
A Lightweight Approach for On-the-Fly Reflectance Estimation
Kihwan Kim, Jinwei Gu, Stephen Tyree, Pavlo Molchanov, Matthias Nießner, Jan Kautz
IEEE International Conference on Computer Vision (ICCV 2017)
Cascaded Scene Flow Prediction using Semantic Segmentation
Zhile Ren, Deqing Sun, Jan Kautz, Erik B. Sudderth
International Conference on 3D Vision
Multiframe Scene Flow with Piecewise Rigid Motion
Vladislav Golyanik, Kihwan Kim, Robert Maier, Matthias Nießner, Didier Stricker, Jan Kautz
IEEE International Conference on 3D Vision (3DV 2017)
Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness
Nikolai Smolyanskiy, Alexey Kamenev, Jeffrey Smith, Stan Birchfield
IROS 2017
Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Networks
Jinwei Gu, Xiaodong Yang, Shalini De Mello, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017
Deep 360 Pilot: Learning a Deep Agent for Piloting through 360 Sports Videos
Hou-Ning Hu, Yen-Chen Lin, Ming-Yu Liu, Hsien-Tzu Cheng, Yung-Ju Chang, Min Sun
CVPR
Production-Level Facial Performance Capture Using Deep Convolutional Neural Networks
Samuli Laine, Tero Karras, Timo Aila, Antti Herva, Shunsuke Saito, Ronald Yu, Hao Li, Jaakko Lehtinen
Symposium on Computer Animation 2017
Polarimetric Multi-view Stereo
Zhaopeng Cui, Jinwei Gu, Boxin Shi, Ping Tan, Jan Kautz
IEEE CVPR 2017
Reconstructing Intensity Images from Binary Spatial Gradient Cameras
Suren Jayasuriya, Orazio Gallo, Jinwei Gu, Timo Aila, Jan Kautz
IEEE Workshop on Embedded Vision (CVPR)
Computational Zoom: A Framework for Post-Capture Image Composition
Abhishek Badki, Orazio Gallo, Jan Kautz, Pradeep Sen
ACM SIGGRAPH
Context-aware Captions from Context-agnostic Supervision
Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, Gal Chechik
Computer Vision and Pattern Recognition
Learning From Noisy Large-Scale Datasets With Minimal Supervision
Andreas Veit, Neil Alldrin, Gal Chechik, Ivan Krasin, Abhinav Gupta, Serge Belongie
Computer Vision and Pattern Recognition

2016

A Patch Memory System For Image Processing and Computer Vision.
Jason Clemons, Chih-Chi Cheng, Iuri Frosio, Daniel Johnson, Steve Keckler
International Symposium on Microarchitecture (MICRO)
Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification
Xiaodong Yang, Pavlo Molchanov, Jan Kautz
ACM Multimedia
Learning to generalize to new compositions in image understanding
Yuval Atzmon, Jonathan Berant, Vahid Kezami, Amir Globerson, Gal Chechik
Arxiv
Reflectance Modeling by Neural Texture Synthesis
Miika Aittala, Timo Aila, Jaakko Lehtinen
ACM Transactions on Graphics 35(4) (proc. SIGGRAPH 2016)
Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks
Pavlo Molchanov, Xiaodong Yang, Shaline Gupta, Kihwan Kim, Stephen Tyree, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016
Accelerated Generative Models for 3D Point Cloud Data
Ben Eckart, Kihwan Kim, Alejandro Troccoli, Alonzo Kelly, Jan Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016

2015

Robust Model-based 3D Head Pose Estimation
Gregory P Meyer, Shalini Gupta, Iuri Frosio, Dikpal Reddy, Jan Kautz
IEEE International Conference on Computer Vision (ICCV) 2015
MLMD: Maximum Likelihood Mixture Decoupling for Fast and Accurate Point Cloud Registration
Ben Eckart, Kihwan Kim, Alejandro Troccoli, Alonzo Kelly, Jan Kautz
IEEE International Conference on 3D Vision (3DV2015)
Retrieving Gray-Level Information from a Binary Sensor and its Application to Gesture Detection
Orazio Gallo, Iuri Frosio, Leonardo Gasparini, Kari Pulli, Massimo Gottardi
IEEE Computer Vision and Pattern Recognition (CVPR 2015), Embedded Vision Works…
Filtering Environment Illumination for Interactive Physically-Based Rendering in Mixed Reality
Soham Uday Mehta, Kihwan Kim, Dawid Pajak, Kari Pulli, Jan Kautz, Ravi Ramamoorthi
Eurographics Symposium on Rendering (EGSR 2015)
Hand Gesture Recognition with 3D Convolutional Neural Networks
Pavlo Molchanov, Shalini Gupta, Kihwan Kim, Jan Kautz
IEEE Computer Vision and Pattern Recognition Workshop (CVPRW) 2015
Winner (1st place) Hand Gesture Recognition Challenge
Camera Re-calibration after Zooming based on Sets of Conics
Iuri Frosio, Cristina Turrini, Alberto Alzati
The Visual Computer
Adaptive Segmentation based on a Learned Quality Metric
Iuri Frosio, Ed Ratner
Proceedings of the 10th International Conference on Computer Vision Theory and …

2014

DT-SLAM: Deferred Triangulation for Robust SLAM
Daniel Herrera C., Kihwan Kim, Juho Kannala, Kari Pulli, Janne Heikkila¨
IEEE International Conference in 3DV (3D Vision)
Addressing System-Level Optimization with OpenVX Graphs
Erik Rainey, Jesse Villareal, Goksel Dedeoglu, Kari Pulli, Thierry Lepley, Frank Brill
10th IEEE Embedded Vision Workshop

2013

WYSIWYG Computational Photography via Viewfinder Editing
Jongmin Baek, Dawid Pająk, Kihwan Kim, Kari Pulli, Marc Levoy
Proc. ACM SIGGRAPH Asia
An Energy Efficient Time-sharing Pyramid Pipeline for Multi-resolution Computer Vision
Qiuling Zhu, Navjot Garg, Yun-Ta Tsai, Kari Pulli
VLSI-SOC
Practical SVBRDF Capture in the Frequency Domain
Miika Aittala, Tim Weyrich, Jaakko Lehtinen
ACM Transactions on Graphics (Proc. SIGGRAPH 2013)

2012

Robust Stereo with Flash and No-flash Image Pairs
Changyin Zhou, Alejandro Troccoli, Kari Pulli
CVPR 2012
Realtime Computer Vision with OpenCV
Kari Pulli, Anatoly Baksheev, Kirill Kornyakov, Victor Eruhimov
Communications of the ACM
Detecting Regions of Interest in Dynamic Scenes with Camera Motions
Kihwan Kim, Dongryeol Lee, Irfan Essa
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2012

2011

Gaussian Process Regression Flow for Analysis of Motion Trajectories
Kihwan Kim, Dongryeol Lee, Irfan Essa
IEEE International Conference on Computer Vision (ICCV) 2011

2010

Point Set Registration: Coherent Point Drift
Andriy Myronenko, Xubo Song
PAMI 2010

Researchers

Adithya Murali
Ali Hatamizadeh
Amrita Mazumdar
Andriy Myronenko
Animesh Garg
Ankit Goyal
Arash Vahdat
Arsalan Mousavian
Balakumar Sundaralingam
Benjamin Eckart
Boris Ivanovic
Bowen Wen
Charles Loop
Chen-Hsuan Lin
Cheng Sun
Chien-Yi Wang
Daguang Xu
De-An Huang
Dieter Fox
Donglai Xiang
Fitsum Reda
Frank Wang
Haggai Maron
Haoyu Yang
Heng Yang
Hongxu Danny Yin
Iuri Frosio
Jacob Munkberg
Jaesung Choe
Yao Lu (Jason)
Jean Kossaifi
Linxi "Jim" Fan
Joohwan Kim
Kaichun Mo
Koki Nagano
Max Zhaoshuo Li
Merlin Nimier-David
Michael Stengel
Min-Hung Chen
Ming-Yu Liu
Orazio Gallo
Pavlo Molchanov
Ravi Ramamoorthi
Rinon Gal
Ruth Rosenholtz
Ryo Hachiuma
Samuli Laine
Seungjun Nah
Shalini De Mello
Song Han
Stephen Tyree
Steve Keckler
Thomas Breuel
Thomas Müller
Tianye Li
Timo Aila
Towaki Takikawa
Tsung-Yi Lin
Tucker Hermans
Umar Iqbal
Valts Blukis
Vinu Joseph
Wonmin Byeon
Xinshuo Weng
Xueting Li
Xun Huang
Ye Yuan
Yin Cui
Yogesh Balaji
Yoni Kasten
Yu-Wei Chao
Yue Wang
Yuke Zhu
Yuval Atzmon
Zekun Hao
Zhiding Yu
Ziyue Xu