Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
be 3-dim, got {}".format(x.ndim) att_output = self.mha(x, x, x, key_padding_mask=src_padding_mask) x = x + self.dropout(self.norm1(att_output)) ff_output = self.ff(x) output = x + self.norm2(ff_output) return output As shown in the diagram, the Encoder actually contains N Encoder blocks or layers,