The main part of my code looks like this.
 def repackage_variable(v, volatile=False):
     return [Variable(torch.from_numpy(h), volatile=volatile).unsqueeze(1) for h in v]

 for k in range(len(minbatches)):
         
         minbatch = minbatches[perm[k]]
         x_padded = utils.make_mask(minbatch)
         x_padded = repackage_variable(x_padded, False)
         x_padded = torch.cat(x_padded, 1)
         T = x_padded.size(0)
         B = x_padded.size(1)
         inp = x_padded[:T-1, :].long()
         target = x_padded[1:, :].long().view(-1, 1)
         if use_cuda:
             inp = inp.cuda()
             target = target.cuda()       

  
         mask = (inp != 0).float().view(-1, 1)
         
         hidden = model.init_hidden(batch)
         model.zero_grad()
         #print(inp.size())
         output, hidden = model(inp, hidden)
         output = output.view(-1, n_vocab)
         
         loss = output.gather(1, target) * mask
         loss = -torch.sum(loss) / torch.sum(mask)
         loss.backward()
         

         optimizer.step()

My question is, at each iteration, since all "Variable"s “inp” and “target” are overwritten, will the model state variables like “self.hiddens” also be overwritten? Does the old computation graph still exist in the next iteration?
nvidia-smi shows that about 6G of memory is used, but I am only testing on batch size of 50, and the length should be at most 200, why would it take up so much memory? And the memory size increases among iterations from time to time, but it could stay the same for a while. Any clues what might be the reason?