got it, thanks for the clarification @sgugger. This is very useful.
The hidden state has shape:
last_hidden_states.numpy().shape
Out[53]: (1, 8, 768)

Which means the very first embedding (position [0][0]) corresponds to the [CLS] token, right? The one people usually use as a vector-representation of the whole sentence?
Thanks!