If you call target = torch.argmax(target, dim=1) on a target tensor of the shape [3, 21, 513, 513], the result will have a shape of [3, 513, 513].
I’m not sure, why your target has now the shape [3, 513, 3].
Could you recheck the shape of the target before applying the torch.argmax operation, please?