Hi all,
I am trying to get an autoencoder to work on my L2 normalized dataset. For now I use the MSE loss which works oke but I noticed that a lot of the reconstructed vectors are not nearly on the unit sphere, which they should be since I know every input vector is on there. Is there any simple solution to penalize or enforce the decoder to generate unit sized vectors? I tried calculating the L2 norm of each item in a batch and dividing the output by that but I am not sure this will give the correct results for the computation graph.
Thanks!