To accumulate gradients across an entire epoch, try something along the lines of the following code:
for e in range(epochs):
    optimizer.zero_grad()
    for i in batches_list:
        out = nn_model(i)
        loss = loss_function(out, actual)
        loss.backward()
        loss_sum += loss.item()
    optimizer.step()     
    loss_list.append(loss_sum / num_train_obs)