1. [Publications](/publications) 2. Self-Supervised Viewpoint Learning From Image Collections # Self-Supervised Viewpoint Learning From Image Collections ![](/sites/default/files/publications/ssv_full%20%281%29.gif) Training deep neural networks to estimate the viewpoint of objects requires large labeled training datasets. However, manually labeling viewpoints is notoriously hard, error-prone, and time-consuming. On the other hand, it is relatively easy to mine many unlabeled images of an object category from the internet, e.g., of cars or faces. We seek to answer the research question of whether such unlabeled collections of in-the-wild images can be successfully utilized to train viewpoint estimation networks for general object categories purely via self-supervision. Self-supervision here refers to the fact that the only true supervisory signal that the network has is the input image itself. We propose a novel learning framework which incorporates an analysis-by-synthesis paradigm to reconstruct images in a viewpoint aware manner with a generative network, along with symmetry and adversarial constraints to successfully supervise our viewpoint estimation network. We show that our approach performs competitively to fully-supervised approaches for several object categories like human faces, cars, buses, and trains. Our work opens up further research in self-supervised viewpoint learning and serves as a robust baseline for it. We open-source our code at . ## Authors Siva Karthik Mustikovela (Heidelberg University) Varun Jampani (Google Research) [Shalini De Mello](/person/shalini-de-mello) [Sifei Liu](/person/sifei-liu) [Umar Iqbal](/person/umar-iqbal) Carsten Rother (Heidelberg University) [Jan Kautz](/person/jan-kautz) ## Publication Date Sunday, June 14, 2020 ## Published in [IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020](http://cvpr2020.thecvf.com/) ## Research Area [Computer Vision](/research-area/computer-vision) [Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence) ## External Links [Code](https://github.com/NVlabs/SSV) ## Uploaded Files [Paper (PDF)](https://research.nvidia.com/sites/default/files/pubs/2020-03_Self-Supervised-Viewpoint-Learning/SSV-CVPR2020.pdf "Open file in new window")8.77 MB [Supplementary (PDF)](https://research.nvidia.com/sites/default/files/pubs/2020-03_Self-Supervised-Viewpoint-Learning/SSV-CVPR2020-Supp.pdf "Open file in new window")1.73 MB [Video](https://research.nvidia.com/sites/default/files/pubs/2020-03_Self-Supervised-Viewpoint-Learning/00322_video.mp4 "Open video in new window")16.7 MB ## Copyright This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to .