1. [Publications](/publications)
2. Key-Locked Rank One Editing for Text-to-Image Personalization
 
 # Key-Locked Rank One Editing for Text-to-Image Personalization

  ![](/sites/default/files/styles/wide/public/publications/Key-Locked%20Rank%20Text%20to%20Image%20Personalization.png?itok=c6kwti2y)

 Text-to-image models (T2I) offer a new level of flexibility by allowing users to guide the creative process through natural language. However, personalizing these models to align with user-provided visual concepts remains a challenging problem. The task of T2I personalization poses multiple hard challenges, such as maintaining high visual fidelity while allowing creative control, combining multiple personalized concepts in a single image, and keeping a small model size. We present *Perfusion*, a T2I personalization method that addresses these challenges using dynamic rank-1 updates to the underlying T2I model. Perfusion avoids overfitting by introducing a new mechanism that "locks" new concepts' cross-attention Keys to their superordinate concept. Additionally, we develop a *gated* rank-1 approach that enables us to control the influence of a learned concept during inference time and to combine multiple concepts. This allows runtime efficient balancing of visual-fidelity and textual-alignment with a single 100KB trained model. Importantly, it can span different operating points across the Pareto front without additional training. We compare our approach to strong baselines and demonstrate its qualitative and quantitative strengths.



 ## Authors



Yoad Tewel (NVIDIA, Tel-Aviv University)

Rinon Gal (NVIDIA)

[Gal Chechik](/person/gal-chechik)

[Yuval Atzmon](/person/yuval-atzmon)

 

 

 ## Publication Date



Sunday, August 6, 2023

 

 ## Published in



[SIGGRAPH 2023](https://s2023.siggraph.org/)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence)

[Computer Graphics](/research-area/computer-graphics)

 

 

 ## External Links



[Arxiv](https://arxiv.org/abs/2305.01644)

[Project Page](https://research.nvidia.com/labs/par/Perfusion/)

 

 

 ## Uploaded Files



[Key\_Locked\_Rank\_One\_Editing\_for\_Text\_to\_Image\_Personalization.pdf](https://d1qx31qr3h6wln.cloudfront.net/publications/Key_Locked_Rank_One_Editing_for_Text_to_Image_Personalization.pdf "Open file in new window")51.62 MB