Bert, NLP.
is a [12 x 768] tensor # Sum the vectors from the last four layers. sum_vec = torch.sum(token[-4:], dim=0) # Use `sum_vec` to represent `token`. token_vecs_sum.append(sum_vec) print ('Shape is: %d x %d' % (len(token_vecs_sum), len(token_vecs_sum[0]))) Shape is: 22 x 768 Sentence Vectors To get a