Yes, sir. Your answer give me great inspiration. A model output in my code was assigned to requires_grad = True, which resulted in accumulation of GPU allocated memory with training step. Thanks for your kindness for answering.