I am also facing the same trouble, did you find any solutions? @ptrblck could you please give some time, if you are available. Thanks in advance.
Specifically, I am facing trouble to understand how to provide padded sequence mask in TransformerEncoderLayer? In TransformerEncoderLayer there are two mask parameters: src_mask and src_key_padding_mask, what will be content(is it boolean or -inf/0) and shape? which parameters is responsible for the sequence padding mask?