If I understand correctly, w is learnable parameter with size EmbSize.
What if you can do the following?
BatchSize = 10
MaxSentenceLength = 20
EmbSize = 30

v1 = torch.randn(BatchSize, MaxSentenceLength, EmbSize)
v2 = torch.randn(BatchSize, EmbSize, MaxSentenceLength)
w = nn.Parameter(torch.randn(1,1,EmbSize))

out = torch.bmm(w * v1, v2)