Hello Kirk!



 kirk86:

Someone suggested me to do a logit transform of the dataset before passing it to the model. Has anyone heard of it before?


I am somewhat skeptical of the idea.  I certainly wouldn’t do it without
also building an equivalent model without the logit transformation and
checking that the logit transformation improves things.


Is the following correct in the sense of performing logit transform to the dataset?
 transform_train = tr.Compose(
            [tr.Resize((32, 32)),  # <-- should I put logit transform here?
...
             tr.Normalize((.5, .5, .5), (.5, .5, .5)),
             lambda x: x + args.sigma * torch.randn_like(x),
             lambda x: -torch.log(1/x - 1)]  # <-- logit transform
        )



Some comments:
Your logit function looks right.
I think you have to wrap your lambda in a pytorch tr.Lambda
transformation.
The logit function maps a probability in [0.0, 1.0] to [-inf, inf].
There is no reason to think that your data reside in [0.0, 1.0],
and, even if they did, the tr.Normalize could well push them outside
of [0.0, 1.0].  (The mean of 0.5 and standard deviation of 0.5 don’t
mean that the result will lie strictly withing [0.0, 1.0]; deviations
of one-and-a-half and two standard deviations are quite likely, so after
tr.Normalize, you are likely to have values outside of [0.0, 1.0].)
If your raw input data were naturally probabilities I think it would make
sense to apply the logit transformation.  But otherwise my intuition is
that doing so is not likely to be helpful.  In any event, the raw data, or
a transformed version of it, had better lie within [0.0, 1.0]; otherwise
the logit transformation will mix things up (it’s not monotone), and will
risk (unnecessary) divergences.
Best.
K. Frank