BERT implements a pooler output because it has been pretrained with an additional objective (next sentence prediction) that uses such output. RoBERTa does not use that objective anymore but it subclasses BERT in code so it also has that functionality. XLNet however is trained on PLM and is different altogether.

Thanks, but I don’t think that fully explains why XLNet doesn’t have a pooler_output. You are right that BERT is trained on next-sentence-prediction, but that only affects the classifier top layer (the linear layer that takes the pooler_output and produces logits). I’m interested in only the pooler_output, the summarization of all the token embeddings. To be consistent with BERT and Roberta, XLNet should produce a pooler_output of its own. I will use that sentence embedding to train my own classifier top layer using a downstream task.