Yes this is not unusual. The first epoch is always the slowest because everything needs to load onto the gpu at first. After that the model and gradients are loaded on the gpu already so it is much faster.