Hey Arman,
Thanks for your input. I am new to transformers. I appreciate your help! In response to recommendations:

This sounds to me like SMOTE. Is that correct? As I understand, SMOTE doesn’t work well with text data. Is that correct?
This is an interesting idea. As I understand, I would do an encoder with a mask and then decode to find the mask. So input would look like this:

[CLS]The patient reports headache and fatigue[SEP]Fatigue, [MASK][SEP]
Wouldn’t this imply to the model that the output has an inherent order to it? (i.e. Fatigue comes before headache)