1. [Publications](/publications)
2. Self-Supervised Object Detection via Generative Image Synthesis
 
 # Self-Supervised Object Detection via Generative Image Synthesis

  ![](/sites/default/files/publications/Picture2_7.gif) 

 We present SSOD – the first end-to-end analysis-by-synthesis framework with controllable GANs for the task of self-supervised object detection. We use collections of real-world images without bounding box annotations to learn to synthesize and detect objects. We leverage controllable GANs to synthesize images with pre-defined object properties and use them to train object detectors. We propose a tight end-to-end coupling of the synthesis and detection networks to optimally train our system. Finally, we also propose a method to optimally adapt SSOD to an intended target data without requiring labels for it. For the task of car detection, on the challenging KITTI and Cityscapes datasets, we show that SSOD outperforms the prior state-of-the-art purely image-based self-supervised object detection method Wetectron. Even without requiring any 3DCAD assets, it also surpasses the state-of-the-art rendering-based method Meta-Sim2. Our work advances the field of self-supervised object detection by introducing a successful new paradigm of using controllable GAN-based image synthesis for it and by significantly improving the base-line accuracy of the task.



 ## Authors



Siva Karthik Mustikovela (University of Heidelberg)

[Shalini De Mello](/person/shalini-de-mello)

Aayush Prakash (NVIDIA)

[Umar Iqbal](/person/umar-iqbal)

[Sifei Liu](/person/sifei-liu)

Thu Nguyen-Phuoc (University of Bath)

Carsten Rother (University of Heidelberg)

[Jan Kautz](/person/jan-kautz)

 

 

 ## Publication Date



Tuesday, October 12, 2021

 

 ## Published in



[International Conference on Computer Vision (ICCV) 2021](https://iccv2021.thecvf.com/)

 

 ## Research Area



[Computer Vision](/research-area/computer-vision)

 

 

 ## Uploaded Files



[Paper (PDF)](https://d1qx31qr3h6wln.cloudfront.net/publications/01764.pdf "Open file in new window")724.8 KB

[Supplementary (PDF)](https://d1qx31qr3h6wln.cloudfront.net/publications/01764-supp.pdf "Open file in new window")6.8 MB

[Video](https://d1qx31qr3h6wln.cloudfront.net/publications/ssod_video.mp4 "Open video in new window")73.74 MB