I want to use gradient accumulation like below:
accum=8
optimizer.zero_grad():
for step in train_loader:
    out=model(input)
    loss=criterion(out,target)
    loss.backwad()
   if step % accum==0:
        optimizer.step()

I  don’t call optimizer.zero_grad() again. What will be? Since my training looks very well.