Hi @koodailar,

(…) we already have a trained internal state in the training (…)

Why do you say so? Have you trained the initial state of the RNN as a model parameter?
In most applications, the internal state is reset (to zero) after each sequence. Having an internal state is useful because it may be used to store information from the “past” that might be useful in the “future”, i.e. information from the previous inputs that conditions the probability distribution of the next outputs. In your example, each input sequence (surname) is completely independent from the following inputs, so it makes sense to reset the state to zero in order to flush the memory.