Sorry, I re-read your original post,
I think instead of using a loss of X - X’, you should use a loss that computes some distance between “embedding” and “inverse embedding”. This will also make more sense in terms of learning.