Are you trying to do Multi Label Classification? If so, use BCELoss, not Multi label margin loss. I will be uploading a tutorial on the same soon, as a lot of people are having problems with multi label.