After batch size reduction, then consider adafactor optimiser instead of Adam. Also if you can get fp16 training to converge successfully for your chosen model then that would be cut memory needs a lot.