Hi @leimao, I am recently working on similar problems and eventually end up in this example: flownet2 3.
It seems using the at::cuda::getCurrentCUDAStream() as the 4th parameter in your kernel will respect the stream setting in your network backbone (i.e. in pytorch code, guarded by with torch.cuda.stream(s):).
I tested it myself using nvidia profiler and it did work.