Depends on the network, the batch size and the GPU you are using.
This link gives some measures 1.1k on torch models (which should be somewhat similar in run-time compared to PyTorch).
As an illustration, in that use-case, VGG16 is 66x slower on a Dual Xeon E5-2630 v3 CPU compared to a Titan X GPU.