Hi @facehugger2020
XLNet and RoBERTa can take sequence pairs just like BERT does. For roberta its’s encoded likes <s>sent1 </s></s> sent2</s>. If you pass the sentence pair to the tokenizer it’ll automatically encode it like that.
You can use the run_glue.py script here 6 to fine-tune XLNet, RoBERTa  on NLI
WIth GPT-2, you can use GPT2Model class (without LM head), and use the final embeddings returned by the model and pass them to the classification layer. Here’s one example 29 I found.