If your label vector is one-hot, using MultiLabelMarginLoss is doing the wrong thing, although it won’t give error. Pytorch convention directly uses indices, so your target should be [batch_size x 1]. With MultiLabelMarginLoss, you are essentially saying that your target labels contain a lot of label 0s and one label 1.