@albanD yes, the optimizer is Adadelta. Fortunately, I found the problem. Each time I finished one iteration, I saved and loaded the best weights using “init.constant_”; replacing it for “copy()” solved the problem.