To be rigorous, I compared (1) BertModel’s pooler output to (2) the output of SequenceSummary for the same sentence. Aren’t these two approaches supposed to produce the same sentence embedding? That’s what I’m led to believe from the comments of SequenceSummary 2:

class transformers.modeling_utils. SequenceSummary
Compute a single vector summary of a sequence hidden states.
summary_type ( str ) – The method to use to make this summary. Accepted values are:


"last" – Take the last token hidden state (like XLNet)

"first" – Take the first token hidden state (like Bert)

"mean" – Take the mean of all tokens hidden states

"cls_index" – Supply a Tensor of classification token position (GPT/GPT-2)

"attn" – Not implemented now, use multi-head attention



Returns
The summary of the sequence hidden states.

I’ve created a Google Colab notebook 8 that computes both (1) pooler output and (2) SequenceSummary output for the same sentence. The last line in the notebook shows that the two approaches to sentence embeddings are not the same.
Is this a bug, or was my asssumption wrong? If it’s a bug (in code or documentation), I’ll open a GitHub issue.
Any help would be appreciated.