I think in that last for you’re building a new full Variable graph through the model for every iteration, none of which actually get freed, since they’re used for the next computation. volatile=True in your Variable constructors should help.