I didn’t read the slides in details, but I still don’t understand why initializing your hidden state to something non-zero (independent of input) would be very different? Within the sequence, the hidden states are still passed through, which is a rather more important thing.