Artificial Intelligence, Deep Learning, Machine Learning, Data Science.
scaling modifies the softmax function, transforming logits (z) into a probability distribution over possible output. Mathematically, the softmax function at temperature is defined as: Eq. 1: Temperature-scaled softmax function where p_i (T) is the probability of the i-th class, z_i is the logit