@lyan62  I’ve used this attention mechanism, in the following paper: https://arxiv.org/abs/1804.06659 165. I will share the source code of our models in this repo https://github.com/cbaziotis/ntua-slp-semeval2018-task3 171 in a few days. In the meantime you can find my implementation of the simple self-attention mechanism here: https://gist.github.com/cbaziotis/94e53bdd6e4852756e0395560ff38aa4 267