Just noticed, this doesnt share the embedding across encoder-input/decoder-input/output?  Per https://arxiv.org/abs/1608.05859 12 might be interesting to do that?  Also, this is used by the ‘attention is all you need’ paper,  https://arxiv.org/abs/1706.03762 4 . Thoughts on how this would be implemented in a pytorch-idiomatic way?  (I can put this into a separate/new thread/topic/question perhaps?)