I think you’d have to test and find out, I can imagine arguments for either way.
Yes, the state is zero-initialized, unless you capture the state, and pass it back in on the next call. See these two docs for examples of manual state management:


  
      

      TensorFlow
  

  
    

Recurrent Neural Networks (RNN) with Keras  |  TensorFlow Core



  

  
    
    
  

  


  
      

      TensorFlow
  

  
    

Neural machine translation with attention  |  Text  | ...