Also, I already know that my 512 outputs are phases meaning the true targets are continuous values between 0 and 2 * pi. Is there any way to add something like an activation function that does the mod 2 * pi calculation so my prediction is always within that range, and is also differentiable? I know tanh is also an option, but that will tend to push most of values at the boundaries.