Solved by quanguet in post #2 
              
              
                Even the definition of SGD is applicable to one data point but in Pytorch, as you said, SGD with batch_size > 1 will be similar to BGD. 
Your code is, I think, okay. However, could you check your code part of optimizer. It seems model.parameters() is missing in SGD initialization. Please try to redu…