XLnet sentence vector
how can i get the XLnet sentence vector by pytorch-transformers. I use the sample but I only get the word vector. it drives me crazy
You can train the model on a downstream task to get a sentence vector related to your task or you can get a sentence vector by averaging or max-pooling the output sequence of token hidden-states.<|||||>Try doing:

```Python
model = model_class.from_pretrained(pretrained_weights,
                                    output_hidden_states=True,
                                    output_attentions=True)

sequence_summary = SequenceSummary(model.config)

es = torch.tensor([tokenizer.encode(s)])
# sentence embedding
t = sequence_summary(model(es)[0])<|||||>A simple strategy is to take the concatentation of the last hidden state, the mean-pooling, and the max-pooling (tends to be a reasonably good baseline pooling strategy, e.g. [ULMFit](https://arxiv.org/pdf/1801.06146.pdf))<|||||>@cpcdoy hey, and I have a question why the sentence is not stationary.<|||||>So, we should pool over all hidden states, and not just use the hidden state corresponding to `[CLS]`?<|||||>I would go with @rishibommasani's solution for a general "semantic" sentence embeddings or fine-tuning and using the `[CLS]` for a task-specific sentence embedding.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
