Hey, thanks for replying!



 futscdav:

I think I know exactly the paper you are trying to implement, and the z distrubution is already supposed to be activated with Softmax. The annealing is just a way to rebalance the final distribution for each pixel to avoid the desaturated effect of taking the mean color of a distribution.


Could you explain more about this? I think that the annealing is used only in the test step to predict the a,b channels. During the training step, the raw values of the feature maps are the input to the Cross Entropy Loss that, under the hood, apply the softmax(). So when I get the raw values of the feature maps in the test step, I need to convert them to probabilities applying the annealing and then softmax(), but the problem is that there are negative values and this break the log() of annealing.



 futscdav:

And for the record, it’s completely expected to have negative values in feature maps, otherwise ReLU would have no effect.


Make total sense hahahah