Hi,
The torch.optim modules gives you access to different types of optimizer to choose the step to take.
For the gradient computation, you have all the freedom to do it in your code. optimizer.zero_grad() can be used to reset the gradients stored for every parameters. Calls to .backward() will accumulate gradients and optimizer.step() will actually apply one step of the given optimizer.
To do batch-gd, you need to zero_grad at the beginning of your epoch, then forward-backward all your samples and at the end do one step.
For a minibatch version, zero_grad, forward-backward a subset of your samples then step and repeat until the end of the epoch.
Finally stochastic GD is the same where you backward a single sample every time.