Right here are two options off the top of my head.

Take the average of all of the output embeddings
Use the CLS embedding (if it is a BERT-ish model)

This will ensure that you always have the same vector size (768, I think)