1. [Publications](/publications)
2. CosAE: Learnable Fourier Series for Image Restoration
 
 # CosAE: Learnable Fourier Series for Image Restoration

  ![](/sites/default/files/styles/wide/public/publications/network.png?itok=dw7b3oyh)

 In this paper, we introduce Cosine Autoencoder (CosAE), a novel, generic Autoencoder that seamlessly leverages the classic Fourier series with a feed-forward neural network. CosAE represents an input image as a series of 2D Cosine time series, each defined by a tuple of learnable frequency and Fourier coefficients. This method stands in contrast to a conventional Autoencoder that often sacrifices detail in their reduced-resolution bottleneck latent spaces. CosAE, however, encodes frequency coefficients, i.e., the amplitudes and phases, in its bottleneck. This encoding enables extreme spatial compression, e.g., 64× downsampled feature maps in the bottleneck, without losing detail upon decoding. We showcase the advantage of CosAE via extensive experiments on flexible-resolution superresolution and blind image restoration, two highly challenging tasks that demand the restoration network to effectively generalize to complex and even unknown image degradations. Our method surpasses state-of-the-art approaches, highlighting its capability to learn a generalizable representation for image restoration. The project page is maintained at <https://sifeiliu.net/CosAE-page/>.



 ## Authors



[Sifei Liu](/person/sifei-liu)

[Shalini De Mello](/person/shalini-de-mello)

[Jan Kautz](/person/jan-kautz)

 

 

 ## Publication Date



Tuesday, December 10, 2024

 

 ## Published in



[Advances in Neural Information Processing Systems (NeurIPS) 2024](https://proceedings.neurips.cc/paper_files/paper/2024/file/13e8be77982beb73d7ed0bbf122f9f3c-Paper-Conference.pdf)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence)

[Computer Vision](/research-area/computer-vision)

 

 

 ## External Links



[Project Page](https://sifeiliu.net/CosAE-page/)

[Video](https://www.youtube.com/watch?v=eYvohDeF8CE)

 

 

 ## Uploaded Files



[Paper](https://d1qx31qr3h6wln.cloudfront.net/publications/NeurIPS-2024-cosae-learnable-fourier-series-for-image-restoration-Paper-Conference.pdf "Open file in new window")48.34 MB