I am sorry to answer you so late, I wasbusy with the project of my tutor recent days.
Here is the output of python -m torch.utils.collect_env

bf16e28fa8edecc349be2fafd90085b850×840 25.6 KB

here is the output when the bug occurs

image1800×35 8.11 KB

here is the training part of my code and the criterion_T is a self-defined loss function in this paper Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
and here is the code of the paper code 1, my criterion_T’s loss is the ‘Truncated-Loss.py’ in that code
the bug occur in the line 
        for image, target, index in tqdm(loader_train, total=int(data_train.__len__()/opt.batch_size)):
        #for image, target, index in loader_train:
            target = target.cuda(opt.cuda_device)
            # Target = F.one_hot(target[2].cuda(opt.cuda_device), opt.n_classes)
            image = image.cuda(opt.cuda_device)

            ########## train resnet1
            resnet.net.zero_grad()
            out1 = resnet.net.forward(image)

            with torch.no_grad():
                cee1 = F.cross_entropy(out1, target, reduce=False, reduction='none')
                L1 = cee1.cpu().numpy()
                ind1 = np.argsort(L1)
                update_num1 = get_update_num(L1[ind1])
                # dataset_all.static_right_label(index[ind[:clean_num]].tolist())
                update_all_num1 += update_num1

            ########## train resnet1
            densenet.net.zero_grad()
            out2 = densenet.net.forward(image)

            with torch.no_grad():
                cee2 = F.cross_entropy(out2, target, reduce=False, reduction='none')
                L2 = cee2.cpu().numpy()
                ind2 = np.argsort(L2)
                update_num2 = get_update_num(L2[ind2])
                # dataset_all.static_right_label(index[ind[:clean_num]].tolist())
                update_all_num2 += update_num2

            loss1 = criterion_T(out1, Target, index)
            loss2 = criterion_T(out2, Target, index)

            resnet.optimizer.step()

            loss2.backward()
            densenet.optimizer.step()```

Finally thank you very much for answer my question, good luck to you.