I had the same problem, and I solved by using with torch.no_grad():
For example,
            # train model 
            input = Variable(data)
            input = input.float()
            target = Variable(label)
            
            if opt.use_gpu:
                input = input.cuda()
                target = target.cuda()

            optimizer.zero_grad()
            with torch.no_grad() :            
                score = model(input)
                loss = criterion(score,target) / (400 * 190)

            loss = Variable(loss, requires_grad = True)
            loss.backward()
            optimizer.step()