This might caused by Oout of memory.
I run the code in debug mode, and step on each line, the gpu memory usage start to increase and will result in
find_frame: frame not found.
    Looking for thread_id:pid_20017_id_140663683239608, frame_id:140663360355304
    Current     thread_id:pid_20017_id_140663683239608, available frames:
    94804309553960  -  94804309571304  -  140648434923752  -  140663064649256  -  94804081775976


    find_frame: frame not found.
    Looking for thread_id:pid_20017_id_140663683239608, frame_id:140663360355304
    Current     thread_id:pid_20017_id_140663683239608, available frames:
    94804309553960  -  94804309571304  -  140648434923752  -  140663064649256  -  94804081775976

RuntimeError: CUDA out of memory. Tried to allocate 600.00 MiB (GPU 0; 10.91 GiB total capacity; 9.78 GiB already allocated; 352.44 MiB free; 205.70 MiB cached)


But it still is weird that if I run this code in run mode, the error just happened in grid_sample().
With help of torch.cuda.memory_allocated(dev), torch.cuda.memory_cached(dev), I can see the allocated and cached memory are 2,733,509,632 and 5,054,136,320 just before execution of grid_sample, then the error RuntimeError: CUDA error: an illegal memory access was encountered occurs.
Still, if use mode=nearest in grid_sample(), no error occurs