Would also suggest check this out:

  
      

      keras.io
  

  
    

Keras documentation: Multi-GPU and distributed training 2



  

  
    
    
  

  


Ideally, you would want to encapsulate your loop inside a subclassed model (tf.keras.Model) by overriding the train_step() method.
Consider the GAN example here. See how the loop is implemented inside train_step().  You could simply initialize the subclassed model within the scope and make use of multiple GPUs. I have done this on multiple occasions and it has only simplified my workflow.