I’m not really sure how an ODE solver looks like (if you could point me to an algorithm that would be great), but as long as you implement it in terms of pytorch operators and keep in mind CUDA performance things (https://pytorch.org/docs/stable/notes/cuda.html# 25) you should see some speedup over the CPU.