From my understanding of character based text generation, we predict the next character based on the previous sequence_len characters.

I think this is different from what we typically mean by text generation.
What we usually mean is that you are given a sequence of chars (x_0, x_1, …x_n), and you would want to generate another seq of chars (y_0, y_1, …y_n). y_0, i.e., the first char in the output seq would depend on the input seq (specifically, the output of the encoder that takes X as the input) but the rest would depend on the previous char (unless you are using attention).