I found Synchronize() on the nvprof even though I did not sync after creating a non-default stream as non-blocking.
Does this happen inside the forward() function of the module?
If so, let me know exactly where it happens on the code.