Indeed, I didn’t thought that changing the gradient parameters and/or strategies (Adadelta/SGD/…), would change the result that much.
Eventually, using Adadelta with defalut parameters worked the best for my problem!
Thanks