My recommendations need to be used both in the training function and in the evaluation function when setting the decode initial state.
Example code:

 # Set initial decoder hidden state to the encoder's final hidden state
if encoder.model_type == 'GRU':
    decoder_hidden = encoder_hidden[:decoder.n_layers]
    
elif encoder.model_type == 'LSTM':
    #Get the encoder final h_hidden_state
    encoder_h_hidden, encoder_c_hidden= encoder_hidden
    decoder_h_hidden = encoder_h_hidden[:decoder.n_layers]
    decoder_c_hidden = encoder_c_hidden[:decoder.n_layers]
    # Recombine the final hidden states as hidden tuple
    decoder_hidden = (decoder_h_hidden, decoder_c_hidden)