Doesn’t the network need to use a fully-connected layer in order for the weights to be updated properly? How would the network be able to update its weights if there is no output to evaluate and give to the loss function? If I understand it correcty, you’re suggesting using the hidden layer’s output to represent the embeddings being provided as input, but it seems the two types of vectors would be generated in a very different way. How would the network “know” to associate one with the other?