Hi,
I am training the model with  input batch size of 16. After each batch iteration the parameter of model increase. why ?
here is my code
model = model.train()
for i in range(0, 335, 16 ):
       y = 0
        for k, x in enumerate(model.parameters()):
            y += x.sum()
        print (y)
        input = data.cuda()
        target =  labels.cuda()
        outputs = model(inputs)
        loss = criterion(outputs, target)
        optimizer.zero_grad()
        loss.backward(retain_graph=True)
        optimizer.step()  
        curr_err = loss.item()
        err += curr_err
        torch.cuda.empty_cache()


here is the increase parameter output
tensor(421.4539, device='cuda:0', grad_fn=<ThAddBackward>)
tensor(422.1019, device='cuda:0', grad_fn=<ThAddBackward>)----]16/335
tensor(423.4700, device='cuda:0', grad_fn=<ThAddBackward>)----]32/335
tensor(424.6810, device='cuda:0', grad_fn=<ThAddBackward>)----]48/335
tensor(425.5378, device='cuda:0', grad_fn=<ThAddBackward>)----]64/335 

......................................  ...........................
................................    ..............................

tensor(434.1239, device='cuda:0', grad_fn=<ThAddBackward>)----]304/335
tensor(434.4677, device='cuda:0', grad_fn=<ThAddBackward>)==>-]320/335

Ideally it should be 421.4539 each time.
please help me out.