Not sure if it will fit your solution but the following tutorial gives you an idea of setting the initial LSTM states of a decoder in an NMT setting:

  
      

      TensorFlow
  

  
    

Neural machine translation with attention  |  Text  | ... 5