Thank you for your reply and thanks for pointing this out!
There is no RoBERTa model in my arget language so I guess I’ll try BERTShare.
To make sure I correctly understand this: BERTShare means both Encoder and Decoder are BERT and they share the same weights right?
That’s great, because cutting the memory footprint is gonna help me out a lot