I new to Pytorch - so please excuse my tender knowledge 
The issue I have is with a LSTM model - I am using a GPU (Nvidia 1060 6GB) - and all models are tasked to use the GPU
However - while running a LSTM  or a RNN model the CPU usage jumps to 50+% and stays there for the duration of the epochs.
This is not the case with GRU or CNN models (CPU steady at 10-12%)
I have checked all the steps and all are running on the GPU (bar the optimiser.step()
It it normal for LSTM and RNN models to cause a CPU spike like this?
I think (pretty sure) the CPU spike is caused by the optimizer.step() call.
Thanks in advance.