1. [Publications](/publications)
2. Neural FFTs for Universal Texture Image Synthesis
 
 # Neural FFTs for Universal Texture Image Synthesis

  ![Publication image](/sites/default/files/styles/wide/public/default_images/default.jpeg?itok=qUFsuJCP "Publication image")

 Synthesizing larger texture images from a smaller exemplar is an important task in graphics and vision. The conventional CNNs, recently adopted for synthesis, require to train and test on the same set of images and fail to generalize to unseen images. This is mainly because those CNNs fully rely on convolutional and upsampling layers that operate locally and not suitable for a task as global as texture synthesis. In this work, inspired by the repetitive nature of texture patterns, we find that texture synthesis can be viewed as (local) \\textit{upsampling} in the Fast Fourier Transform (FFT) domain. However, FFT of natural images exhibits high dynamic range and lacks local correlations. Therefore, to train CNNs we design a framework to perform FFT upsampling in feature space using deformable convolutions. Such design allows our framework to generalize to unseen images, and synthesize textures in a single pass. Extensive evaluations confirm that our method achieves state-of-the-art performance both quantitatively and qualitatively.



 ## Authors



[Morteza Mardani](/person/morteza-mardani)

Guilin Liu (NVIDIA )

Aysegul Dundar (NVIDIA)

Shiqiu Liu (NVIDIA)

Andrew Tao (NVIDIA)

Bryan Catanzaro (NVIDIA)

 

 

 ## Publication Date



Saturday, December 5, 2020

 

 ## Published in



[NeurIPS 2020](https://proceedings.neurips.cc/paper/2020/hash/a23156abfd4a114c35b930b836064e8b-Abstract.html)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence)

[Computer Vision](/research-area/computer-vision)

[Generative AI](/research-area/generative-ai)

 

 

 ## Uploaded Files



[NeurIPS-2020-neural-ffts-for-universal-texture-image-synthesis-Paper.pdf](https://d1qx31qr3h6wln.cloudfront.net/publications/NeurIPS-2020-neural-ffts-for-universal-texture-image-synthesis-Paper.pdf "Open file in new window")8.96 MB