Thank you for the reply!
However, my question is that I see some codes in pytorch which they use batch size greater than 1 with SGD and this means it is not SGD really anymore. So, how does pytorch calculate SGD when batch size is greater than 1?