Rafael_Valle:

in 0.4 the memory usage increases on every variable addition, producing OOM after a few iterations


Just to make sure, iterations here mean training iterations, right? That is, graph is freed after each iteration (usually via loss.backward()).