rwightman:

The geometric mean of the ‘probabilities’ from the softmax layer provide the best result for this.


Hi Ross,
do you know if there is a fundamental argument for using the Geometric mean rather than the arithmetic or is it more empirical?
If I assume that the prediction is a noisy estimate of the “true PD” with sufficiently nice noise, my intuition would lead me to an arithmetic estimate. Also, the geometric mean of probabilities doesn’t sum to 1 (in general), so you’d need to re-normalize:
a = torch.randn(2,4)
p = a.softmax(1)
p_gm = (p[0]**0.5*p[1]**0.5)
print(p_gm.sum())

Best regards
Thomas