Dear @albanD , after experimenting a bit I still have some doubts:

I have used torchviz as suggested but in my case it is not helping. I have 2 “models”: one proper model which is my network, and one two dimensional nn.Parameter to store the learnt losses weights.

You suggested to check if the graph is growing in size with torchviz, but model is working properly and not growing in size. When inspecting the nn.Parameters, torchviz only shows one box with none.
However it seems that torchviz is of no help for visualizing the autograd backward graph. The only method I found online is get_grad(), but it is verbose.

  
      stackoverflow.com
  
  
      
    
  

  How can I visualize what happens during loss.backward()? 8



  pytorch, backpropagation



  asked by
  
  
    Tom
  
  on 12:05PM - 25 Oct 18 UTC


  
  
    
    
  
  


How can I visualize and understand the backward graph to understand what is generating out of memory? It seems very hard to inspect what it going on under the hood, mostly in unconventional applications such as gradNorm.
Many thanks,
Stefano