I want to get sentences’ embedding vectors for other classification tasks
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
model = BertModel.from_pretrained("bert-base-uncased")
inputs = tokenizer('this is a test.', return_tensors="pt")
outputs = model(**inputs)

If I do this way:
embedding_of_sentence = outputs[1]
Here, according to the documentation, the outputs[1] is the:
* **pooler_output** ( torch.FloatTensorof shape(batch_size, hidden_size) ) – Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer and a Tanh activation function. The Linear layer weights are trained from the next sentence prediction (classification) objective during pretraining.
The last layer hidden state of the first token CLS of the sentence for classification, which seems right.
However, in another post, they are suggesting using “usually only take the hidden states of the [CLS] token of the last layer”,
  

      github.com/huggingface/transformers
  

  
    
  
	  
  

  
    
      word or sentence embedding from BERT model 60
    

    
      
        opened 
        
          
        
        Nov 26, 2019
      
      

        
          closed 
        
          
        
        Aug 3, 2020
      
        

      
        
          
          ghost
        
      
    

    
        wontfix
    
  


  
    How can I extract embeddings for a sentence or a set of words directly from pre-…trained models (Standard BERT)? For example, I am using Spacy for this purpose at the moment where I can do it as follows:

sentence vector:
`sentence_vector = bert_model("This is an apple").vector`

word_vectors:
```
words = bert_model("This is an apple")
word_vectors = [w.vector for w in words]
```

I am wondering if this is possible directly with huggingface pre-trained models (especially BERT).
  

  

  
    
    
  

  


and the code is:
embedding_of_last_layer = outputs[0][0]
embedding_of_sentence =embedding_of_last_layer[0]

The results from the two methods for the sentence embedding are different. Which one is right or better?