I want to regress some coordinates from images and these coordinates were normalized to [-1,1] in my data set. When I just use a linear layer at the end of my network, I very often get NaN as output of my L1 or MSE loss right in the first episode. I guess this is because of exploding gradients or something similar. This happens both with learning rates 0.1 and 1.0. I am using SGD with momentum=0.9 and nesterov=True.
When I use tanh instead of just a linear layer in the end, this does not seem to happen. However, I am unsure if this is an optimal solution because when you look at the tanh graph, it is much more sensitive to changes that are close to x=0 than to changes at x > 1 or x < -1.  So if I have output values close to 1 or -1, the input to tanh has to be very big. I hope I have explained my doubts well enough.
What are recommended methods to ensure that the regression output is always in a specific range, in my case [-1,1]?
Additional information: Before the regression layer in the end I have a lot of convolutions and deconvolutions, but I think that this should not be the problem here.