Oh applying LBFGS is interesting.
In fminunc3 5, i can set the max iteration.
So I set it 400.
Umm… Maybe there are some differences between stochastic gradient descent (SGD) algorithm and regular Gradient Descent. I couldn’t find regular Gradient Descent in Pytorch, that’s why I am using SGD even if I apply all batch at one potimization not using mini batch. And It went different.