Hi Yuri!



 yuri:

The question now how does this mathematically work?


The formula for probabilistic mode is given the the CrossEntropyLoss
documentation.


especially that Target contains float with arbitrary positive values not just between 0 and 1


As you have recognized, it is possible to pass in invalid values for
target in the probabilistic case, namely values that do not represent
a valid (discrete) probability distribution.  That is, individual values can
lie outside of [0.0, 1.0], or the values can sum to a value other than
1.0.
CrossEntropyLoss doesn’t validate this condition – it simply applies
the formula given in the documentation and will not return a sensible
result if you violate it.  (For example, if some of the target values are
negative, you could get a negative loss that could cause your training
to diverge.)  It’s up to you to pass in valid target values.
Best.
K. Frank