From a theoretical point of view, achieving a learnable hidden state initialization while doing truncated BPTT is not valid, because gradients are truncated and thus cannot be backpropagated to the original initialization.