Hi,
Option 2 is indeed the best. To train a multi-label classifier, you can use an xxxForSequenceClassification model (which is a Transformer encoder with a linear layer on top), and set the problem_type attribute of the configuration to multi_label_classification. For example, if you want to use BERT, you can do it as follows:
from transformers import BertForSequenceClassification

model = BertForSequenceClassification.from_pretrained("bert-base-uncased", problem_type="multi_label_classification")

As you can see in the code 22, it will use the BCE (binary cross-entropy) loss.
Note that if the number of labels you have is > 2, you also need to specify num_labels=... when calling the .from_pretrained() method.