Deep Learning, Sora, Computer Vision, Transformers, Attention.
(4, 480, 768).the attention matrix will have shape of (4, 480, 480). so we need to expand our attention mask along the x-direction so that is is also of the shape (4, 480, 480). below is an image, we can see that 3 & 4 images in the batch has some sequences masked with zeros. mask_ =