It seems you are returning a model when you call self._train_epoches. Assuming that this function applies your optimizer to the gradients of the parameters you want to train, given your loss, maybe you don’t need to return anything since model is a reference, simply remove the return value of self._train_epoches(…) and update the model from within that function? Probably what is happening is you are adding a new model’s worth of memory each time this function is called, i.e. making a copy of the model and not freeing the previous copy’s memory. Just a guess.
Also if you are going to call del on variables I would follow it up with gc.collect() since sometimes I find this doesn’t really clear the memory, but rather just deletes the variables name-binding.