Thanks, @albanD !
So, the variables no longer referenced will be freed by using torch.cuda.empty_cache() right?
And I started to locate the memory consuming objects, but I couldn’t locate what variables gave me a bad effect.
As for variables related to cuda, I use same variable name (e.g. model, criterion) in different trials.
Anyway, I suspected below evaluation loop (sorry, it’s version 0.4, maybe version causes the problem?),
for i, (X, y) in tqdm(enumerate(val_loader), total=len(val_loader)):
    X = Variable(X.cuda())
    y = Variable(y.squeeze().cuda(non_blocking=False))

    with torch.no_grad():
        outputs = model(X)
        loss = criterion(outputs, y)

    prec1, prec5 = accuracy(outputs.data, target, top_k=(1, 5))
    losses.update(loss.data[0], X.size(0))
    top1.update(prec1[0], X.size(0))
    top5.update(prec5[0], X.size(0))