Have you looked at sentence-transformers?

Yes, I’m doing research in the same area.

I am not sure what you mean with “taking into account attention masks”, why do you need those?

BERT will insert [PAD] tokens as needed. The attention masks mark where those tokens are located in the input sequencer. I don’t want to include the embedding for the those tokens when I compute the max or the mean.