Bert, NLP.
single vector for our entire sentence we have multiple application-dependent strategies, but a simple approach is to average the second to last hidden layer of each token producing a single 768 length vector. # `hidden_states` has shape [13 x 1 x 22 x 768] # `token_vecs` is a tensor with shape [22