Ah interesting. What is defined as same? To the same variable with same shape/dtype?
I did this because my profiler was showing the .cuda calls as the bottleneck but I think there is an issue with the profiler so I’ll try to update to 0.4 to try your pytorch profiler.