CUDA calls are asynchronous, so your current Timer class will most likely only time the Python overhead and the kernel launch.
To properly time the kernels, you would have to synchronize the code before starting and stopping the timer via torch.cuda.synchronize().