KFrank:

Note, that in the case that hist.min() == 0 (perfectly possible), all
of your weights will be zero. I would eliminate m = hist.min() , and
go with 1.0 / v (in place of m  / v ).


I checked the code again and realised this situation could happen which I have not foreseen. Thanks for the headsup @KFrank  But since the following is still true



 Bendrien:

my labels are exclusive. There has to be always exactly one true of all three classes.


I fixed this issue by removing all zeros before taking the minimum: m = hist[hist != 0].min()



 Bendrien:

I have had to use the retain_graph parameter of the backpropagation and I found the documentation on this feature a bit sparse. Do I use this correct and could someone explain to me whats happing there in more detail and what data will be retained?


This article got me covered:

  
      
      Medium – 11 Sep 19
  
  
    

PyTorch Autograd 2

Understanding the heart of PyTorch’s magic

  Reading time: 8 min read