Solved by chenyuntc in post #4 
              
              
                To me, the logsigmoid+NLLLoss combination hardly makes any sense, because the objective function only tries to promote the gt_class, but no suppression on the negative ones. Maybe you wanna try sigmoid+bceloss.