Hi @vasudevgupta (appreciate I’m cheekily @-ing you!)
The above runs fine, but the number of dimensions output are different depending on the length of the text.
For example if i use the string 'hello' and call .shape on the tensor returned, the dims are ([1,3,768])
But if I use 'hello my dog is cute', the dims are [(1,7,768)].
Perhaps this is expected behaviour? I have experience using various language models in tensorflow and they tend to generate vectors with the same numbers of dimensions. Perhaps there is something really fundamental or obvious I’m missing here?
For context, I’m exploring matching conversational type questions to knowledge base articles, and I wanted to test this language model for suitability. So I’m trying to calculate cosine similarity between embeddings of questions and those of articles.
Any help would be much appreciated! My guess is I have to use one of the hidden layers as embeddings but I’m not sure.