What is an intuitive interpretation for the softmax transformation? A recent question on this site asked about the intuition of softmax regression.  This has inspired me to ask a corresponding question about the intuitive meaning of the softmax transformation itself.  The general scaled form of the softmax function $\mathbf{S}: \bar{\mathbb{R}}^{n-1} \times \mathbb{R}_+ \rightarrow \Delta^n$ is given by:
$$\mathbf{S}(\mathbf{z}, \lambda) \equiv \Bigg( \frac{1}{1 + \sum_{k=1}^{n-1} \exp(\lambda z_k)}, \frac{\exp(\lambda z_1)}{1 + \sum_{k=1}^{n-1} \exp(\lambda z_k)}, \ \cdots \ , \frac{\exp(\lambda z_{n-1})}{1 + \sum_{k=1}^{n-1} \exp(\lambda z_k)} \Bigg).$$
Is there any intuitive interpretation that describes the mapping from the extended-real value $\mathbf{z} \in \bar{\mathbb{R}}^{n-1}$ to the corresponding probability vector $\mathbf{p} \in \Delta^n$ in simple terms?  I.e., is there some explanation that can describe the mapping in an intuitive way by use of geometry, analogy, etc.?

One way to think about the softmax function is that it gives you an output that can be interpreted as a probability distribution (i.e., all numbers are in the range [0,1], and they sum to 1).  This is useful, because then the output of the softmax can be interpreted as a "probability" of each class/category (conditioned on the features).
Why does its output always have this property? Well, the softmax is essentially the composition of two steps:

*

*Apply the exp function to each value.  This makes all values non-negative.

*Normalize the values so they sum to 1 (by dividing by the sum).  This makes all values sum to 1.

After both of these steps, you are guaranteed that all values are non-negative and they sum to 1, which means they can be interpreted as a probability distribution.
The generalized softmax with scaling $\lambda$ just amounts to multiplying all values by $\lambda$, then applying the softmax, so it is not very different from the normal softmax.
Another way to think about the softmax is that it is a natural generalization of the standard logistic function $f(x) = e^x/(1+e^x)$, used in logistic regression.  Logistic regression is used when you want to do two-class classification.  When you want to do multi-class classification, you replace the standard logistic function with the softmax function.  If you apply the softmax function with two classes, the result reduces to the standard logistic function that you're used to in (two-class) logistic regression.
