wiem_chakroun:

BUT the GPU memory usage increase from one epoch to other.


That’s good to know.
Are you storing the loss or some other output of your model somewhere?
For debugging or printing purposes you should detach() the outputs or store the loss.item() e.g. in a loss.
Otherwise, you will store the whole computation graph, which might eventually yield an out of memory error and might also slow down your code.
Could this be the case in your code?