1. [Publications](/publications) 2. Importance Estimation for Neural Network Pruning # Importance Estimation for Neural Network Pruning ![](/sites/default/files/styles/wide/public/publications/resnet_result.png?itok=Eu6YkCVb) Structural pruning of neural network parameters reduces computation, energy, and memory transfer costs during inference. We propose a novel method that estimates the contribution of a neuron (filter) to the final loss and iteratively removes those with smaller scores. We describe two variations of our method using the first and second-order Taylor expansions to approximate a filter's contribution. Both methods scale consistently across any network layer without requiring per-layer sensitivity analysis and can be applied to any kind of layer, including skip connections. For modern networks trained on ImageNet, we measured experimentally a high (>93%) correlation between the contribution computed by our methods and a reliable estimate of the true importance. Pruning with the proposed methods leads to an improvement over state-of-the-art in terms of accuracy, FLOPs, and parameter reduction. On ResNet-101, we achieve a 40% FLOPS reduction by removing 30% of the parameters, with a loss of 0.02% in the top-1 accuracy on ImageNet. Code is available at [https://github.com/NVlabs/Taylor\_pruning](https://github.com/NVlabs/Taylor_pruning) ## Authors [Pavlo Molchanov](/person/pavlo-molchanov) Arun Mallya (NVIDIA) [Stephen Tyree](/person/stephen-tyree) [Iuri Frosio](/person/iuri-frosio) [Jan Kautz](/person/jan-kautz) ## Publication Date Wednesday, June 12, 2019 ## Published in CVPR2019 ## Research Area [Artificial Intelligence and Machine Learning ](/research-area/machine-learning-artificial-intelligence) ## External Links [Paper PDF](https://arxiv.org/abs/1906.10771) ## Copyright This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to .