I don’t think the hidden layer’s output had to be put through an additional transformation. Usually, implementations related to machine translation, image captioning or similar realms learn a linear transformation, as a second to last step (pre-SoftMax-ing) to map the hidden size to the vocabulary dimensionality (for choosing the most probable prediction).
However, if your objective is to use similar_by_vector for finding the nearest word representation in the vocabulary, I feel you wouldn’t need any additional layers or SoftMax on top of the standard LSTM network.