I see. I assumed you were comparing both on the CPU for some reason. Regarding GPU, have you subtracted the time it takes to transfer data from main memory to the GPU?
Plus, your operations don’t seem to be well suited for taking advantage of GPUs. You want to compare things like matrix multiplications etc. where you have parallelism and is based on CUDA and cuDNN stuff more heavily.