Yep that’s right - BERTShare (and the other “share” models) have shared weights for the encoder & decoder. Now what I don’t know is whether you can initialise this in the EncoderDecoder class or not.
Perhaps the resident seq2seq experts (@valhalla or @patrickvonplaten) can help answer this: does EncoderDecoder support the BERTShare model variants from the Leveraging Pre-trained Checkpoints for Sequence Generation Tasks paper?