I try to fine-tune the resnet152 for multi-label classification where the number of labels is 1024.

   multiLabelLoss = th.nn.MultiLabelSoftMarginLoss()  
   predict = resnet(img)                                                                                                                                                                
   loss = multiLabelLoss(predict, label)    


where label is batch_size x n_category and so is predict
The loss decreases quite well from 0.7 to 0.04. However, the accuracy on the training set is very low about 0.09%.
I compute the accuracy using the hamming loss and implemented as

 predict = th.nn.functional.sigmoid(predict) > 0.5  
 r = (predict == label.byte())  
 acc = r.sum().data[0] 
 acc = float(acc) / 1024


How could this happen? Is there any mistake using the MultiLabelSoftMarginLoss ?