Thanks a lot.
And What I mean is for mini-batch gradient descent, My first code is to use batch_size examples, and I use the criterion to calculate average loss.
My second code is also to use batch_size examples. and the difference is that every time i calculate loss of example of batch_size examples, (means loss_1, loss_2, and loss_batch_size), finally, I add them and average. Then I use backward and optimizer to update.
If I do not use batch normalization or some other which may influence the loss. May be the result is the same?
Thanks for your reply