@ecolss creating optimizer is nearly free (order of ms), especially that you do it at most once per epoch, and you only often have only few epochs in jobs that take a number of days to complete. Don’t optimize prematurely.