Hi Sylvain,
Glad to hear from you, outside of FastAI  Well, I am here as “Beginner” and will have to study more about Trainer. In the meantime, I tried the following:

run BertForSequenceClassification as usual
Take out logits from output (discard the loss from Bert run)
calculate new loss from nn.CrossEntropyLoss
and then calculate loss.backward()

Model runs okay, but I am not sure if this is a legitimate approach…
Thanks.