Hey Frank



 KFrank:

Note, that in the case that hist.min() == 0 (perfectly possible), all
of your weights will be zero. I would eliminate m = hist.min() , and
go with 1.0 / v (in place of m  / v ).


I think hist.min() cant be zero since my labels are exclusive. There has to be always exactly one true of all three classes. And for the normalisation I just chose 1 as it is the most common value in those situations.



 KFrank:

I would probably just calculate the weights for the entire training set,
and not bother doing it on a per-batch basis.


I was wondering about that too and decided to recalculate on a per batch basis for those cases in which my sequence does not have one class to appear at all. As far as I understand this will make the true meaning of this sequence more clear and granular for the backpropagation.
As a side note, when I’m doing my evalutation on test data I reset all weights to 1, since this would be the normal usecase of the model in the wild - this makes sense, right?
Here is one followup question 
During training I have sequences of variable length. For not having unnecessary zero-padding within my CNN I want to feed one sequence as a whole to my model. But for backpropagation I thought it would make sense to compute the losses on a fixed window to make the loss computation independent of the incoming sequence length. So i came up with this routine (its a simplified version ignoring the weight computation):
inputs, labels = sequence_data
outputs = model(inputs)

for l in range(0, len(labels), window_size):
    r = l + window_size
    loss = criterion(outputs[l:r], labels[l:r])
    optimizer.zero_grad()
    retain_graph = r < len(labels)
    loss.backward(retain_graph=retain_graph)
    optimizer.step()

For this to work I have had to use the retain_graph parameter of the backpropagation and I found the documentation on this feature a bit sparse. Do I use this correct and could someone explain to me whats happing there in more detail and what data will be retained?