It is probably a pretty stupid question, but I am not able to get to the solution.
I am using torch.no_grad() during inference to avoid keeping track of the history. The problem is that the memory increases like 5GB during this stage. I am only able to solve it using .data. This was the classical way I solve it, however I will like to move to the new approaches.
   #memory increases a lot
    with torch.no_grad():
                for idx,(x,t) in enumerate(data_valid):                                                   
                        x,t=x.cuda(),t.cuda()
                        out=net.forward_test(x)
                        MC_valid+=net.classification_error(out,t)

   #memory is oK
    for idx,(x,t) in enumerate(data_valid):                                                   
               x,t=x.cuda(),t.cuda()
               out=net.forward_test(x)
               MC_valid+=net.classification_error(out,t).data