Thanks for your reply.
I am aware that those characters would also be taking into account for the prediction, however my concern is more related to the error signal during training and the evaluation during testing. In the current approach there is only one backpropagation every bptt characters and similarly for testing, perplexity is only estimated on a fraction of characters in the test set.