Sure, the question is a little bit ambiguous but this:



 falmasri:

for example Adam


lead me to think that @falmasri was referring to same optimizer instances.
Having different mathematical optimizers remains a interesting problem! And it should be addressed with empirical observations, as you said.