I am training a deep network and because the training data is very big, so I can’t feed model with a bigger batch size. I wander if I can feed each time with a small batch size and optimize the model once after a specified times of loss backward?
If possible, how can i do to average the gradient values before update parameters?
Thank you!!!