Might be. What model are you using at the moment?
Also, it seems you are storing the computation graph in this line of code:
accumulated_loss += loss#scaled_loss

If you don’t need to call backward on accumulated_loss, make sure to use accumulated_loss += loss.detach().