Hello, let’s say that i have an embedding matrix: nn.Embedding(3, 5)
I want to average these embeddings, so first i have to sum their elements.

question.PNG815×140 7.09 KB

If i use torch.sum(my_embeddings, dim=0) or add them one by one i get the same result but with different backward operations as you can see in the screenshot. What is the best option for this or is it the same?