Oh, thank you for your reply! I used to think “.cuda()” is just to store the parameter on GPU, and now I find you are right!
I modified my code and left it alone, added “model.cuda()” before training, I think maybe the question mentioned above has been solved while another had arisen. Well, do you have any idea what that means?

Traceback (most recent call last):
File “cifar_bi.py”, line 370, in 
main()
File “cifar_bi.py”, line 203, in main
train_loss, train_acc = train(trainloader, model, criterion, optimizer, epoch, use_cuda)
File “cifar_bi.py”, line 267, in train
loss.backward()
File “/home/xcz/.local/lib/python3.5/site-packages/torch/tensor.py”, line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File “/home/xcz/.local/lib/python3.5/site-packages/torch/autograd/init.py”, line 89, in backward
allow_unreachable=True)  # allow_unreachable flag
RuntimeError: leaf variable has been moved into the graph interior

Anyway, thank you for answering my question!