Thanks Ken, thats a nice example, i got the intuition now!
One quick question, i think i should also pass the the y variable through a linear layer since there is W_1 multiplying it in the original function.

$$ c_i =  W_1( tanh(W_2m_i + W_3v_a + b_i )$$

You think this is necesary or just applying the softmax straight away is ok?
def forward(self, m_i, v_a):
        y1 = self.lin1(m_i)
        y2 = self.lin2(v_a)
        y = F.tanh(y1+y2)
        y = self.lin3(y)
        y = F.softmax(y)
        return y