Solved by dpernes in post #2 
              
              
                Just found the issue! My function get_accuracy() was returning a variable accuracy instead of the tensor accuracy.data. Since the return value of this function is accumulated in every training iteration (at train_accuracy += get_accuracy(tag_scores, targets)), the memory usage was increasing immense…