Body: <p>I am pruning a neural network (CNN and Dense) and for different sparsity levels, I have different sub-networks. Say for sparsity levels of 20%, 40%, 60% and 80%, I have 4 different sub-networks.</p>
  
  <p>Now, I want to find the non-zero connections that they have in common. Any idea how to visualize this or compute this?</p>
  
  <p>I am using Python 3.7 and TensorFlow 2.0.</p>
  
  <p>After the convergence of a neural network following the random weight initialization, some weights/connections increase (magnitude), while other weights decrease. You can then prune the smallest magnitude weights. I want to compare the remaining weights for say two networks having the same level of sparsity of say 50%. The idea is to have an idea of which weights were pruned away and which weights/connections remain.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-21T22:01:12.190Z
Comments.Text: The paper [Visualizing the Hidden Activity of Artificial Neural Networks](https://www.researchgate.net/publication/306049229_Visualizing_the_Hidden_Activity_of_Artificial_Neural_Networks) (by Rauber et al.) could be useful. I haven't yet read it. If I read it, I may provide an answer to this question later.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-22T01:44:06.543Z
Comments.Text: edited question to add more information
Comments.UserDisplayName: Arun
Tags: <neural-networks><convolutional-neural-networks><deep-neural-networks>
Title: How can I find the similar non-zero connections between different levels of sparsity of the same network?
UserDisplayName: Arun
ThreadText: # How can I find the similar non-zero connections between different levels of sparsity of the same network?
  
  (Asked by: username_0 on 2020-05-21)
  
  I am pruning a neural network (CNN and Dense) and for different sparsity levels, I have different sub-networks. Say for sparsity levels of 20%, 40%, 60% and 80%, I have 4 different sub-networks.
  
  Now, I want to find the non-zero connections that they have in common. Any idea how to visualize this or compute this?
  
  I am using Python 3.7 and TensorFlow 2.0.
  
  After the convergence of a neural network following the random weight initialization, some weights/connections increase (magnitude), while other weights decrease. You can then prune the smallest magnitude weights. I want to compare the remaining weights for say two networks having the same level of sparsity of say 50%. The idea is to have an idea of which weights were pruned away and which weights/connections remain.
  
  **Tags:** neural-networks, convolutional-neural-networks, deep-neural-networks
  > **Comments:**
  > **username_1**: The paper [Visualizing the Hidden Activity of Artificial Neural Networks](https://www.researchgate.net/publication/306049229_Visualizing_the_Hidden_Activity_of_Artificial_Neural_Networks) (by Rauber et al.) could be useful. I haven't yet read it. If I read it, I may provide an answer to this question later.  
  > **username_0**: edited question to add more information  
  
  ---
  
  ## 0 answers
  
  