Gradient accumulation can be used to simulate larger batch sizes. However, especially if you are using batchnorm (or potentially other norm layers with running stats) you would have to be careful about the internal running stats, as each forward pass would update them with the (noisy) batch statistics (you might need to play around with the momentum).
Also, if you are using a single sample, the batchnorm layers might not be able to calculate the batch statistics and could yield an error.