smu226:

denom = torch.sqrt(denom.sum()/len(denom))


I think it’s this lin  that causes the issue. Can you create a tensor for the len(denom) and moved that to CUDA, something like this ... / torch.tesnor(len(denom)).to(device)