Hi @n8henrie thanks for having tackled this issue already. I was wondering if your code above considers only the case in which padded tokens are not needed? If so in order to extend it to work for padded tokens, I would need to change the mean operation right such that it ignores the encoder embeddings of the padded tokens, right?