I don’t understand. Are you trying to learn the targets? Why do the targets come from an embedding?
If you really want to back propagate to the targets you can just compute the MSE loss by:
loss = ((input - target)**2).mean()