Solved by Hong in post #2 
              
              
                You may need two embedding matrices, one for token, one for pos tags. And the network is like this: 
input token index -> token embedding matrices -> token embedding 
input pos-tag index -> pos-tag embedding matrices -> pos-tag embedding 
and then concatenate token embedding and its corresponding po…