I solved this issue by map_location='cpu’
Loading pretrained model
ckpt = torch.load(path)

automatically allocates the parameters to GPU:0.
I cannot understand why until now, but it is solved by using map_location while I load the model.
ckpt = torch.load(path, map_location='cpu')
model_A.load_state_dict(ckpt['model_state_dict '])
model_A.to(device)