I am getting this error when I try to access grid values that are out of range, in cuda but not in cpu.
x = torch.randn(1, 3, 64, 64)
flow = torch.nn.Tanh()(torch.randn(1, 64, 64, 2))
pred_x = F.grid_sample(x, flow)  # no problem
pred_x = F.grid_sample(x, 2*flow)  # no problem
pred_x = F.grid_sample(x.to(torch.device('cuda')), flow.to(torch.device('cuda'))) # no problem
pred_x = F.grid_sample(x.to(torch.device('cuda')), 2*flow.to(torch.device('cuda')))
# RuntimeError: CUDA error: an illegal memory access was encountered

I am using Pytorch 1.4.0.