Bert, NLP.
vectors, with shape [22 x 3,072] token_vecs_cat = [] # `token_embeddings` is a [22 x 12 x 768] tensor. # For each token in the sentence... for token in token_embeddings: # `token` is a [12 x 768] tensor # Concatenate the vectors (that is, append them together) from the last # four layers. # Each