In a RNN you can use the last hidden layer as a vector representation of a sequence. I was wondering if there is any similar idea in the transformer architecture?
Background:
I am building a model that deals with protein data. At one stage I wish to include raw protein sequence information to the model. For this I need to encode a sequence into a fixed lenght vector that contains information about the sequence on the global scope. This vector will then be concatenated to some other vector and used in downstrema processing.