sbelharbi:

in all the results above, torch.utils.benchmark is set to true.


torch.utils.benchmark is a utility function used to profile operations and is not a flag, so setting it to True won’t do anything besides overriding it.
Based on your description, you are getting the right profiling by synchronizing before starting and stopping the timers and get the expected wrong results otherwise.