you can define your own way (\eg average pool all token vectors) or simply using the BERT default setup, the [CLS] token’s output vector

  
      jalammar.github.io
  
  
    

A Visual Guide to Using BERT for the First Time 12

Translations: Chinese, Russian


  
  


Progress has been rapidly accelerating in machine learning models that process language over the last couple of years. This progress has left the research lab and started powering some of the leading digital...


  
  
    
    
  
  


bert-distilbert-tokenization-11705×922 63.1 KB