I am currently working on implementing Attention for a Variation auto encoder, as implemented in https://joao-pereira.pt/docs/accepted_version_ICMLA18.pdf
I am having a hard time identifying an efficient solution for the following.
I have a tensor A with dimension (11000, 11000)
I also have a tensor H with dimension (11000, 20)
I want to populate a tensor C with the following:
The dimension of C should be (11000, 20)
Where the rows in C are:
C[j,:] = A[0,j] * H[0,:] + A[1,j] * H[1,:] + … + A[11000,j] * H[11000,:]
for all j in range(1,11000)
I tried double for-loops but it was tediously slow.
Hope you can provide some help, thanks in advance!
Best Regards, Jonas