Deep Learning, Sora, Computer Vision, Transformers, Attention.
embeddings we followed Tancik et al. (2020) and used Fourier embeddings. Specifically, coordinates (x, y) were mapped using a single linear layer before applying sin and cos activations to them, and stacking the results to obtained the positional embeddings. fractional encoding we take a slightly