Thanks for the update.
Where is ptr coming  from and did you make sure that the allocated memory is still alive and on the GPU?
This post 3 shows how to use cv::cuda::GpuMat as the input to torch::from_blob.