Are you saving some tensors somewhere (e.g. the loss in a list)?
Often memory leaks are created by trying to store some training information like the loss without detaching it from the computation graph, which will store the whole graph with it.