I have a network with a linear head followed by a sigmoid activation function. Most of my output neurons should be in a range 0-1 and thus sigmoid makes sense, but some of the neurons should output normalized (standardized) colors.
So for the last three neurons I do torch.logit(x) as they can be in the range [-1.3, 1.4]. I still encounter errors sometimes because some of the color neurons output 1.0, and torch.logit(1.0) gives inf.
Should I use something else other than torch.logit() after my activation functions for these special neurons? Maybe normalize (not standardize!) such that the range [0, 1] becomes [-1.3, 1.4]?