1. [Publications](/publications)
2. On the Distance between Two Neural Networks and the Stability of Learning
 
 # On the Distance between Two Neural Networks and the Stability of Learning

  ![](/sites/default/files/styles/wide/public/publications/Screen%20Shot%202021-11-18%20at%206.19.40%20PM.png?itok=ftbudh7Z)

 This paper relates parameter distance to gradient breakdown for a broad class of nonlinear compositional functions. The analysis leads to a new distance function called deep relative trust and a descent lemma for neural networks. Since the resulting learning rule seems to require little to no learning rate tuning, it may unlock a simpler workflow for training deeper and more complex neural networks. The Python code used in this paper is available at [this URL](https://github.com/jxbz/fromage).



 ## Authors



Jeremy Bernstein

[Arash Vahdat](/person/arash-vahdat)

Yisong Yue

[Ming-Yu Liu](/person/ming-yu-liu)

 

 

 ## Publication Date



Sunday, February 9, 2020

 

 ## Published in



[Neural Information Processing Systems (NeurIPS) 2020](https://arxiv.org/abs/2002.03432)

 

 ## Research Area



[Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence)