Hi Beilei!
The short answer is that you are mixing gpu and cpu tensors together.
(But I do not know why commenting out act1 and act2 makes your
error (appear to) go away.)



 111220:

images=torch.rand((4,3,300,300))
images=images.cuda().float()



Your images live in the gpu.


labels=torch.from_numpy(np.array([1,0,0,1]))



Your labels live in the cpu.


model=myNet()
model=model.cuda()



Your model lives in the gpu.


global_feat,local_feat,cls_score=model(images)



Gpu images passed through gpu model.  Okay.


cls_score=cls_score.to('cpu')



cls_score now lives in the cpu.


labels=one_hot_smooth_label(labels,20)



labels still live in the cpu because one_hot_smooth_label() returns
a cpu tensor.


criterion=nn.BCEWithLogitsLoss()
loss=criterion(cls_score,labels)



Cpu cls_score and cpu labels passed to BCEWithLogitsLoss.
Okay.  (And loss lives in the cpu.)


loss.backward()

RuntimeError: Function AddBackward0 returned an invalid gradient at index 1 - expected type torch.FloatTensor but got torch.cuda.FloatTensor



But you are trying to backpropagate a cpu loss through a gpu model.
So somewhere inside, your tensors don’t match.


#if we delete act1 and act2 in model,the error will not happen



(I don’t know why this “fixes” the error, but I doubt it’s the real issue.)


How can I solve this problem？


Probably the simplest thing would be to leave cls_score in the gpu,
and move labels to the gpu.  Thus:
# cls_score=cls_score.to('cpu')
labels=one_hot_smooth_label(labels,20)
labels = labels.cuda()
criterion=nn.BCEWithLogitsLoss()
loss=criterion(cls_score,labels)
loss.backward()

Now cls_score, labels, and loss should all be consistently in the
gpu, so you should be able to backpropagate your gpu loss through
your gpu model.
Good luck.
K. Frank