Hi there,
if you want a randomly initialized decoder then you can create the decoder separately, save it, and then pass that to the from_encoder_decoder_pretrained. Following code snipper shows how you can init the encoder using pre-tarined bert-base and a randomly initialized decoder of bert-base size.
from transformers import BertConfig, BertLMHeadModel, EncoderDecoderModel

decoder_config = BertConfig(is_decoder=True)# bert-base size
decoder = BertLMHeadModel(decoder_config)
decoder.save_pretrained("decoder") # save the decoder

model = EncoderDecoderModel.from_encoder_decoder_pretrained("bert-base-uncased", "decoder")

# verify the decoder
model.decoder.config.is_decoder
model.decoder.config.add_cross_attention