Thanks! Is it right that forward() calculates the gradients for parameters with require_grad=True, and backward() will accumulate those? If so, it makes sense that backward() increases with batch size. But it still doesn’t make sense that step() also takes longer… Any idea?