You can also take a look at this example that shows how to retrieve the current state of an RNN layer like LSTM and pass that to initialize another:

  
      

      TensorFlow
  

  
    

Neural machine translation with attention  |  Text  | ... 2