I’m trying to implement the shared log_softmax found in this architecture for question-answering: http://aclweb.org/anthology/P18-1078 2, in section 3.3. This is the relevant formula:

Screenshot from 2018-10-06 17-40-33.png522×660 157 KB

How would I go about implementing it in PyTorch in away that’s stable and differentiable? Is there any existing function that I can leverage?