I guess my question is if you should be careful about the time you are going to wrap a Tensor in a Variable, for example, because you may still want to manipulate the data in  it, and it is in a Variable, it may keep track of these operations and do something unexpected during backward. Do I have to be more careful handling variables than tensors?
In the following example, memory was leaking with each call to train, but it stopped leaking when I wrapped tensors in variables only when calling model and margin_loss, rather than once in the beginning. Is this a bug or am I overlooking something?
def train(model, sents, tags, optimizer, kpred):
    margin_loss = nn.MultiMarginLoss(margin=1)
    
    model.train()
    for (s, t) in itertools.izip(sents, tags):
        len_t = t.size(0)
        t_pred = torch.LongTensor(len_t)
        t_pred[:kpred] = t[:kpred]
        s = Variable( s )
        t = Variable( t )
        t_pred = Variable( t_pred )
    
        for idx in xrange(kpred, len_t):
            optimizer.zero_grad()
            scores = model(s, t_pred, idx)
            loss = margin_loss(scores, t[idx:idx + 1] )
            loss.backward()
            optimizer.step()

            _, ti_pred = torch.max(scores, 1)
            t_pred[idx] = ti_pred[0, 0]
            t_pred[idx] = t[idx]

Thanks