i dont think it’s a memory. You might be creating reference cycles, and python cant deallocate your Variables without a garbage collection pass.
Try adding this line to your training for loop:
import gc
gc.collect()

Let me know if that works.