Bert, NLP.
on the FAQ page of the project. bert-as-service, by default, uses the outputs from the second-to-last layer of the model. This is the summary of Han’s perspective : The embeddings start out in the first layer as having no contextual information (i.e., the meaning of the initial ‘bank’ embedding