Training time vs inference (“real world”) time have many different features.
Training time for starters is longer because we are exposing the model to considerably more samples – multiple times → a dataset set size of 10,000 samples for 10 epochs is 100,000 operations of the data being passed through the model. In addition at training time we are doing calculations on the backward pass (to train the model) – this takes times. Luckily GPUs allow us to parallelize much of this
At inference time you are only working with one (or a few samples) and the model is only being exposed to that one time – we also have the benefit of not needing to do some work on the backward pass. Depending on how big your inference batch is - oftentimes a CPU may suffice – making the real world problem much more tenable