I am totally new to this NLP and transformer and attention.
I was playing with models sentence-transformers and want to explore more, but now I got stuck.
I have an input of BxKx768 which is my embedded features.
Is there a way to give them to a transformer(which has attention model ) and get the output of size BxM where M can be any number?
I learned to do it when input is a sentence, but have no idea how to do it when input is features.
I guess what I am asking is how to give my input to a transform model and have my output.
Apologies in advance if it is a bad question.
If that is easy, are there different models to try, like e.g. in resnet we have resnet 18,50,etc, do have the same thing here?