This is spot on. If you are timing your model using CPU timers (like whatever is built into Python), then your timings will not reflect GPU times accurately, since kernels are executed asynchronously on the GPU.