Hello Yash!



 yash1994:

I am computing loss in the following manner:
def calculate_loss(output, target, criterion):
    loss = 0
    for i in range(output.size(1)):
        loss += criterion(output[:, i, :], target[:,i])
    return loss # / output.size(1)



As an aside, you can use the “K-dimensional case” feature of
CrossEntropyLoss (if you’re using a recent enough version of
pytorch) to eliminate your for loop.
output.size (1) * torch.nn.CrossEntropyLoss() (output.transpose (1, 2), target)

should yield the same result as your calculate_loss function.


Questions:

Is this kind of loss calculation is permissible?



Yes, there is nothing in principle wrong with what you are doing.



How does the backdrop happen in this scenario?



The same as usual.  output is a differentiable function of your model
parameters, the loss in your loop for each chunk is a differentiable
function of output, and summing loss over the four chunks is
differentiable, so the gradients backpropagate through your summed
loss all the way back to your model parameters.


My model trains well for a few batches (150/2500) but then loss and accuracy both stagnate. And when I check the output of each chunk it produces homogeneous classification values (almost same class), regardless of the target having different classes in each chunk.


Many things could cause this, but it’s not caused directly by your
“chunked-loss” function.  There could be some bug in your model,
or it could just be that the problem you’re working on is hard (or
impossible), and the way your training data and model interact
makes training difficult.
Best.
K. Frank