As I understand effective batch size = batch grad accumulation * batch size.
Firstly
Is this the only adjustment that needs to be done, nothing for the learning rate?
Secondly how much can you accumulate. Say I want a batch size of 256, and can only fit a batch size of 4, can I really accumulate gradients over 64 batches or is that way too much and never done in practice?