1. [Publications](/publications)
2. Pretraining codomain attention neural operators for solving multiphysics pdes
 
 # Pretraining codomain attention neural operators for solving multiphysics pdes

  ![Publication image](/sites/default/files/styles/wide/public/default_images/default.jpeg?itok=qUFsuJCP "Publication image")

 Existing neural operator architectures face challenges when solving multiphysics problems with coupled partial differential equations (PDEs) due to complex geometries, interactions between physical variables, and the limited amounts of high-resolution training data. To address these issues, we propose Codomain Attention Neural Operator (CoDA-NO), which tokenizes functions along the codomain or channel space, enabling self-supervised learning or pretraining of multiple PDE systems. Specifically, we extend positional encoding, self-attention, and normalization layers to function spaces. CoDA-NO can learn representations of different PDE systems with a single model. We evaluate CoDA-NO's potential as a backbone for learning multiphysics PDEs over multiple systems by considering few-shot learning settings. On complex downstream tasks with limited data, such as fluid flow simulations, fluid-structure interactions, and Rayleigh-Bénard convection, we found CoDA-NO to outperform existing methods by over 36%.



 ## Authors



Md Ashiqur Rahman (Purdue University)

Robert Joseph George (Caltech)

Mogab Elleithy (Caltech)

Daniel Leibovici (Caltech)

Zongyi Li (Caltech)

[Boris Bonev](/person/boris-bonev)

Colin White (Caltech)

Julius Berner (Caltech)

Raymond A. Yeh (Purdue University)

[Jean Kossaifi](/person/jean-kossaifi)

Kamyar Azizzadenesheli (NVIDIA)

Anima Anandkumar (Caltech)

 

 

 ## Publication Date



Monday, December 16, 2024

 

 ## Published in



[NeurIPS Proceedings](https://proceedings.neurips.cc/paper_files/paper/2024/hash/bc75fa9843a7905bbed9d83895a88f7f-Abstract-Conference.html)

 

 ## Research Area



[Algorithms and Numerical Methods](/research-area/algorithms)

[Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence)

[Physical AI](/research-area/physical-ai)