The output to be used for getting sentence embeddings from BERT
What is the output that we should be using to get embeddings for a sentence using BERT? When I load the pre-trained BERT model ([BertModel](https://huggingface.co/transformers/model_doc/bert.html#transformers.BertModel)) from huggingface for inference, should I be using the `pooler_output', the output of the last hidden layer or something else?

While fine-tuning BERT, which huggingface module should be used for getting sentence embeddings? Is it the [BertForSequenceClassification](https://huggingface.co/transformers/model_doc/bert.html#bertforsequenceclassification),  [BertForMaskedLM](https://huggingface.co/transformers/model_doc/bert.html#bertformaskedlm), [BertModel](https://huggingface.co/transformers/model_doc/bert.html#bertmodel), or some other module? 
Hi @AkshitaJha , what is your downstream task ? 
Also you may wanna try this out for sentence embeddings 
https://huggingface.co/deepset/sentence_bert
https://github.com/UKPLab/sentence-transformers
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
