if you do N optimizer ‘steps’, your forward() will use N different sets of parameter values, it is not an accumulation.
You also have to scale loss or gradients or LR if doing accumulation, to have an equivalent effect with batches.